Image-stream windowing system and method

ABSTRACT

An image-stream windowing method includes capturing, with a camera located at a fixed position and having a fixed field of view, a high-resolution image stream of an object that moves during said capturing. The high-resolution image stream includes a sequence of high-resolution frames. The method also includes determining, for each high-resolution frame of the sequence of high-resolution frames, a respective window, of a sequence of windows corresponding to the sequence of high-resolution frames, that encloses the object within said each high-resolution frame. The size and location of the respective window are determined based at least on the fixed position, the fixed field of view, and a position of the object. The method also includes generating a low-resolution image stream from the high-resolution image stream by cropping said each high-resolution frame with its respective window.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/884,344, filed May 27, 2020, which is a continuation of U.S. patentapplication Ser. No. 15/811,466, filed Nov. 13, 2017 and now U.S. Pat.No. 10,701,322, which is a divisional of U.S. patent application Ser.No. 11/950,346, filed Dec. 4, 2007 and now U.S. Pat. No. 9,848,172,which claims priority to U.S. Provisional Patent Application No.60/872,639, filed Dec. 4, 2006 and titled “Systems and Processes forCapturing Image Data of Tracked Objects”. Each of these applications isincorporated herein by reference in its entirety.

BACKGROUND

Traditionally, video and still images of a live event (i.e., videocontent and still image content) are created by a team of professionals.In the case of video content, highly trained camera persons operate oneor more cameras and highly trained production staff operate productionequipment (e.g., within a production van at a sporting event) to selectcamera shots and combine graphics into a production feed. In the case ofstill image content, highly skilled camera persons operate still camerasto capture still images of an event and submit these still images to oneor more editors who select shots for use in magazines, for example.

The cost of producing video and still image content defines the marketsize required to cover this cost. Thus, only events having a sufficientmarket justify the cost of producing video and still image content.Although technology has reduced the cost of production, the cost ofskilled human operators remains high.

Images from a camera may be used to visually track an object (e.g., agolf ball) within the camera's field of view. The camera may bemotorized to allow it to move to maintain the moving object within itsfield of view. However, such systems fail when the camera ‘loses sight’of the object; for example, the camera may lose sight of the object ifthe object becomes visually obscured by another object.

For certain sporting events, cameras may be motorized to facilitatetracking of competitors and are operated by remote camera operator.These cameras still require the skill of a person.

Many systems have been developed to track objects by attaching a sensorto the object and then using the sensor to determine the location of theobject. Such object tracking provides data (e.g., speed) to computersystems but is not known to facilitate real image production.

SUMMARY

Systems and processes described hereinbelow provide for autonomous stilland moving picture production.

In one embodiment, a process for capturing images of tracked objectsincludes: assigning each of one or more cameras to each tracked object,and determining a location of each tracked object within an operationalfield. For each of the one or more cameras, a field of view isdetermined, from the location of the camera to the determined locationof the object assigned to the camera. Each of the one or more cameras iscommanded to capture the associated field of view. The steps ofdetermining and commanding are repeated periodically.

In one embodiment, a process for capturing images of tracked objectsincludes utilizing a plurality of location units to determine locationinformation of each of the tracked objects within an operational field.Each of the location units is attached to a different tracked object.For each of one or more cameras, a field of view is determined, from thelocation of the camera to the determined location of each of the trackedobjects. The optimum field of view for each tracked object isdetermined, and one of the fields of view from the location of thecamera is assigned to each of the one or more cameras, based upon thedetermined optimum fields of view. Each of the one or more cameras iscontrolled to capture an image stream of the field of view assigned tothe camera. The steps of utilizing, determining, assigning andcontrolling are repeated periodically.

In one embodiment, a process for controlling a first camera to capture afirst image stream of a first tracked object within an operational fieldincludes: receiving first location information periodically from a firstlocation unit attached to the first object within the operational field,and determining a first field of view for the first camera based uponthe first location information and the location of the first camera. Thefirst camera is controlled to capture the image stream based upon thefirst field of view; and the first field of view and the first cameraare updated continuously as the first location information changes.

In one embodiment, a system for controlling one or more cameras tocapture images of one or more objects includes at least one motorizedcamera device and at least one object location unit attached to each ofthe objects. An object tracking device utilizes the at least one objectlocation unit to determine location information of each of the one ormore objects. A camera control device determines a field of view foreach of the one or more motorized cameras based upon the locationinformation and the location of each of the one or more motorizedcameras. The camera control device controls each of the at least onemotorized cameras to capture images of the associated field of view. Thecamera control device updates each field of view as the locationinformation changes and controls each motorized camera to capture theassociated field of view. Each motorized camera includes motorizedcontrol of rotation, tilt and zoom.

In an embodiment, an autonomous picture production system includes: alocation unit attached to each tracked object; an object tracking devicefor receiving location information from each location unit; at least onemotorized camera; and a camera control device for controlling, basedupon the location information, the at least one motorized camera tocapture image data of at least one tracked object.

In another embodiment, an autonomous picture production system includes:two or more fixed cameras for capturing image data including at leastone tracked object; an object tracking device for determining locationinformation of the at least one tracked object based upon the imagedata; at least one motorized camera, and a camera control device forcontrolling, based upon the location information, the at least onemotorized camera to capture image data of at least one tracked object.

In another embodiment, a process provides autonomous picture production,including the steps of: attaching at least one location unit to eachtracked object at an operational field; determining location of each ofthe tracked objects within the operational field based upon informationreceived from the location units; controlling at least one camera suchthat at least one tracked object is positioned within a field of view ofthe camera; capturing image data from the at least one camera, andrepeating the steps of determining, controlling and capturing as the atleast one tracked object moves, to maintain the tracked object withinthe field of view.

In another embodiment, a picture production process captures image dataof tracked objects, and includes the steps of: selecting one object froma plurality of objects within an operational field in response to inputfrom a user interface, each object having at least one location unit;receiving location information from the at least one location unitattached to the one object; determining a field of view to include theone object from a camera, based upon the location information and thelocation of the camera, and controlling the camera to capture the fieldof view as image data.

In another embodiment, a system provides autonomous picture production,and includes: one or more motorized cameras; at least one location unitattached to each objects to be tracked; an object tracking device fordetermining location of each of the objects to be tracked based uponlocation information obtained from the location units, and a cameracontrol device for determining, for at least one of the one or moremotorized cameras, a field of view that includes one of the objects tobe tracked based upon the location information and a location of each ofthe one or more motorized cameras. The camera control device controlsthe one or more motorized cameras to capture image data of the field ofview.

In another embodiment, a method stores image data in a self-organizingdatabase to facilitate autonomous picture production, including thesteps of: receiving an image and associated annotation data; attachingtags to the image based upon the associated annotation data;categorizing the image based on the attached tags, and storing the imagein the database based on the categorization.

In another embodiment, a camera produces a standard resolution and rateimage stream and a slow-motion image stream of an action of interest tofacilitate autonomous picture production. The camera includes an imagerfor capturing an image stream at a frame rate of the slow-motion imagestream and at a resolution equal to the maximum resolution of thestandard image stream and the slow-motion image stream, and a resolutiondown-sampler for reducing the resolution of each frame of the capturedimage stream if the captured image stream has a higher resolution thanthe slow-motion image stream. A slow-motion buffer stores theslow-motion image stream. A rate and resolution down-sampler reduces theframe rate of the image stream where the frame rate of the image streamis greater than the frame rate of the standard resolution and rate imagestream, and reduces the resolution of each frame of the image streamwhere the resolution of the image stream is greater than the resolutionof the standard resolution and rate image stream, to produce thestandard resolution and rate image stream.

In another embodiment, a camera produces a standard resolution and rateimage stream and still images of an action of interest to facilitateautonomous picture production. The camera includes an imager forcapturing an image stream at a frame rate of the standard resolution andrate image stream and at a resolution equal to the resolution of thestill images. A rate down-sampler reduces the frame rate of the capturedimage stream to a desired still image rate to produce the still images.A still image buffer stores the still images. A rate and resolutiondown-sampler reduces the frame rate of the captured image stream wherethe frame rate of the image stream is greater than the frame rate of thestandard resolution and rate image stream, and reduces the resolution ofeach frame of the captured image stream where the resolution of thecaptured image stream is greater than the resolution of the standardresolution and rate image stream, to produce the standard resolution andrate image stream.

In another embodiment, a camera produces a standard resolution and rateimage stream and a slow-motion image stream of a previously occurredaction of interest to facilitate autonomous picture production. Thecamera has an imager for capturing an image stream at a frame rate ofthe slow-motion image stream and at a resolution equal to the maximumresolution of the standard resolution image stream and the slow-motionimage stream. A resolution down-sampler produces a continuousslow-motion image stream, the resolution down-sampler reducing theresolution of each frame of the captured image stream if the capturedimage stream has a higher resolution than the slow-motion image stream.A first circular buffer continually stores the continuous slow-motionimage stream. A slow-motion buffer stores the slow-motion image stream,the first circular buffer transferring the slow-motion image stream tothe slow-motion buffer upon notification of the previously occurredaction of interest. A rate and resolution down-sampler produces thestandard resolution and rate image stream, reducing the frame rate ofthe captured image stream if the frame rate of the captured image streamis greater than the frame rate of the standard resolution and rate imagestream. The rate and resolution down sampler reduces the resolution ofeach frame of the captured image stream if the resolution of thecaptured image stream is greater than the resolution of the standardresolution and rate image stream.

In another embodiment, a camera produces a standard resolution and rateimage stream and still images of a previously occurred action ofinterest to facilitate autonomous picture production, and includes animager for capturing an image stream at a frame rate of the standardresolution and rate image stream and at a resolution equal to theresolution of the still images. A rate down-sampler continually producesstill images, reducing the frame rate of the captured image stream to adesired still image rate. A first circular buffer stores the continuousstill images. A still image buffer stores the still images of thepreviously occurred action of interest. A rate and resolutiondown-sampler reduces the frame rate of the captured image stream wherethe frame rate of the captured image stream is greater than the framerate of the standard resolution and rate image stream, The rate andresolution down-sampler reduces the resolution of each frame of thecaptured image stream where the resolution of the captured image streamis greater than the resolution of the standard resolution and rate imagestream, to produce the standard resolution and rate image stream.

In another embodiment, an autonomous picture production systemautomatically captures an image of a location upon request, andincludes: one or more motorized cameras; an external interaction devicefor receiving the request from a requestor, the request specifying thelocation, and a camera control device for determining an optimal camerafrom the one or more motorized cameras for capturing the image. Thecamera control device controls the optimal camera to capture the image.A database for stores the image. The external interaction device informsthe requestor how to retrieve the image from the database.

In another embodiment, a method automatically captures an image of alocation upon request, and includes the steps of: receiving a requestfrom a requestor to capture the image of the location; determining anoptimal camera for capturing the image of the location from at least onemotorized camera; controlling the optimal camera to include the locationwithin its field of view; capturing the image using the optimal camera,and delivering the image to the requestor.

In another embodiment, an autonomous picture production process includesthe steps of: automatically determining location of one or more objectsin or adjacent to an operational field, and automatically controllingone or more cameras in response to the location to capture image data ofthe objects.

In another embodiment, a camera facilitates autonomous pictureproduction, and includes: an imager for capturing an image stream; asignal processor for processing the image stream into one or more imagedata paths; at least one image stream output, and a memory forcyclically buffering images of each image data path and for bufferingone or more output image streams of the camera.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a system for capturing image data of tracked objects, tofacilitate autonomous still and/or moving picture production, in accordwith an embodiment.

FIG. 2 shows a system for capturing image data of tracked objects movingwithin a circuit, to facilitate autonomous still and/or moving pictureproduction, in accord with an embodiment.

FIG. 3 shows a system for capturing image data of one or more snowathletes performing within a half-pipe, to facilitate autonomous stilland/or moving picture production, in accord with an embodiment.

FIG. 4 shows a system for capturing image data of grouped trackedobjects, to facilitate autonomous still and/or moving pictureproduction, in accord with an embodiment.

FIG. 5 shows a system that uses a mobile camera to capture image data oftracked objects, to facilitate autonomous still and/or moving pictureproduction, in accord with an embodiment.

FIG. 6 shows a system for capturing unobstructed image data of trackedobjects, to facilitate autonomous still and/or moving pictureproduction, in accord with an embodiment.

FIG. 7 illustrates exemplary control zones within an operational field.

FIG. 8 illustrates an exemplary control zone within an operational fieldthat includes a running track.

FIGS. 9, 10 and 11 are perspective views of exemplary control tracesmade within control zones to provide additional information to cameracontrol devices.

FIG. 12 shows an exemplary process for capturing image data of trackedobjects in autonomous still and/or moving picture production, in accordwith an embodiment.

FIG. 13 shows an exemplary process for capturing unobstructed image dataof tracked objects in autonomous still and/or moving picture production,in accord with an embodiment.

FIG. 14 shows an exemplary system for capturing image data of trackedobjects and determining events of interest related to the trackedobjects, to facilitate autonomous still and/or moving pictureproduction, in accord with an embodiment.

FIG. 15 shows an exemplary system for storing image data of trackedobjects in a self-organizing database.

FIG. 16 shows one exemplary process for storing image data of trackedobjects in a self-organizing database.

FIG. 17 shows an exemplary system for capturing high resolution imagedata of tracked objects interspersed within an image stream, tofacilitate autonomous still and/or moving picture production, in accordwith an embodiment.

FIG. 18 shows the chronology of capturing an action of interest,according to an embodiment.

FIG. 19 shows an exemplary system for displaying image data of trackedobjects in autonomous still and/or moving picture production, in accordwith an embodiment.

FIG. 20 is a schematic diagram of a camera controlled by a cameracontrol device, in an embodiment.

FIG. 21 is a schematic diagram of a camera for buffering still imagesand slow-motion image stream, in an embodiment.

FIG. 22 is a schematic diagram of a production control device, a cameracontrol device and two cameras, in an embodiment.

FIG. 23 is a schematic diagram illustrating one exemplary stadiumhosting a sporting event on a field, in an embodiment.

FIG. 24 shows an exemplary central control that may representintelligence of the camera control devices of FIGS. 1, 2, 3, 4, 5, 6,14, 15, 17, 20, 21, 22 and 23 and production control devices of FIGS. 6,14, 17, 22 and 23.

FIG. 25 shows an exemplary system for including commentary with a videofeed, in an embodiment.

FIG. 26 is a flowchart illustrating an exemplary method for capturingimages of a location upon request, in an embodiment.

FIG. 27 shows an exemplary system for automatically adding commentary toan automatically produced video feed, in an embodiment.

FIG. 28 shows an exemplary system for capturing image data of objectstracked using two fixed cameras, to facilitate autonomous still and/ormoving picture production, in accord with an embodiment.

FIG. 29 is a high-level block diagram illustrating exemplary hardware ofobject tracking devices, camera control devices, cameras, productioncontrol devices and database control devices of FIGS. 1, 2, 3, 4, 5, 6,14, 15, 17, 20, 21, 22 and 23.

FIGS. 30-34 are flowcharts illustrating one exemplary method andsub-methods for capturing a standard image feed, high resolution stillimages and a slow-motion feed using the camera of FIG. 20.

FIG. 35 is a plan view of an operational field with four fixed cameraspositioned at corners of the operational field, each having a fixedfield of view to capture images of activities within the operationalfield.

FIG. 36 shows one exemplary perspective view from one camera of FIG. 35.

DETAILED DESCRIPTION OF THE FIGURES

FIG. 1 shows a system 100 that captures image data of tracked objects.Image data may include images and/or image streams. Image streams arefor example collected data bits that are converted into a signal (e.g.,a video signal sent via a camera interface), which is then convertedback into a data stream (e.g., a stream of bytes), which in turn may beinterpreted as an image or images, for example via processing software.Thus, “image stream” as used herein refers to image data collected by acamera, for example a continuous series of data that is convertible intoa media stream, video for viewing or recording, or high-resolutionimages for viewing or printing. The terms “image data” and “imagestream” are sometimes used interchangeably in the following description.

System 100 has an object tracking device 102 that determines locationsof one or more objects 106 within an operational field 108 and a cameracontrol device 104 that controls one or more cameras 110 to captureimage data of the one or more of objects 106. This image data isillustratively shown being output as live feeds (such as signalsrepresenting the captured image data), indicated by arrow 105. Cameras110 are for example each mounted upon a motorized camera controlplatform that allows remote control of camera functionality (e.g., zoom,brightness, focus, etc.) as well as directional positioning (e.g.,forward, backward and lateral motion, along with pan and tilt) of thecamera.

Object tracking device 102 may interact with a location unit 112, fittedto each object 106 to be tracked, to determine coordinate data 116(i.e., location information) for each object 106. As described ingreater detail below, this coordinate data 116 is for example referencedto a two- or three-dimensional coordinate model (e.g., using Cartesiancoordinates x, y, z). As shown in FIG. 1, location unit 112(1) isattached to object 106(1); location device 112(2) is attached to object106(2); location device 112(3) is attached to object 106(3); andlocation device 112(4) is attached to object 106(4). Accordingly,coordinate data 116(1) is determined for object 106(1) with locationdevice 112(1); coordinate data 116(2) is determined for object 106(2)using location device 112(2); coordinate data 116(3) is determined forobject 106(3) using location device 112(3), and coordinate data 116(4)id determined for object 106(4) using location device 112(4). Coordinatedata 116(1)-116(4) for respective objects 106(1)-106(4) may be sent fromobject tracking device 102 to camera control device 104 viacommunication link 118. Additional or fewer objects 106 may be similarlytracked without departing from the scope hereof.

In an embodiment, location unit 112 is a GPS based location device thatdetermines absolute location and transmits the absolute location toobject tracking device 102. In another embodiment, location unit 112 isa transponder that interacts with one or more transceivers (not shown)of object tracking device 102, to triangulate a position of locationunits 112 within operational field 108.

Operational field 108 may be represented as a two-dimensional modelwithin one or both of object tracking device 102 and camera controldevice 104 so that each determined location of objects 106 isrepresentable by a two-dimensional coordinate of x and y, where xrepresents a longitudinal displacement of object 106 within theoperational field and y represent a latitudinal displacement of object106 within the operational field. The two-dimensional coordinate mayinclude a time stamp that indicates a time at which the x, y coordinatewas determined (e.g., coordinate data 116 may include x, y and t (time)values).

Operational field 108 may be represented as a three-dimensional modelwithin one or both of object tracking device 102 and camera controldevice 104. Each determined location of objects 106 is thusrepresentable by a three-dimensional coordinate of x, y and z, where xrepresents a longitudinal displacement of object 106 with operationalfield 108, y represents a latitudinal displacement of object 106 withinoperational field 108 and z represents a vertical displacement of object106 within operational field 108. This three-dimensional coordinate mayinclude a time stamp that indicates a time when the coordinate wasdetermined (e.g., coordinate data 116 may include x, y, z and t (time)values).

Coordinate data 116 may include information identifying a correspondinglocation unit 112; camera control device 104 may therefore use thisinformation to distinguish between coordinate data for each trackedobject 106 within operational field 108. For example, coordinate data116(2) includes position and identity of location unit 112(2) and cameracontrol device 104 uses this information to track object 106(2) with anassigned camera (e.g., camera 110(2)).

Coordinate data 116 may also include a velocity component that indicatesa direction and speed of a location unit 112. In an embodiment, cameracontrol device 104 uses this component to extrapolate positions ofobjects 106 within operational field 108, to predict fields of view andcamera movements that appropriately maintain objects 106 within thefields of view. Velocity may also be determined where coordinate data116 does not include a velocity component, for example by integratinglocation change over time.

A user interface 114 communicatively couples with camera control device104 to facilitate control of cameras 110. A user (e.g. an operator ofsystem 100) may for example assign a camera 110 to track an object 106,through user interface 114. Illustratively, as shown in FIG. 1, camera110(1) is assigned to object 106(1) and has a field of view 120(1) thatincludes object 106(1); and camera 110(2) is assigned to object 106(3)and has a field of view 120(2) that includes object 106(3). Objectassignments may be made via voice command, key press or touch-screenoptions of user interface 114. Assignment of cameras to objects may alsobe made automatically, as described below. In an embodiment, aproduction team selects from live feed outputs 105 from each camera 110to produce desired content (e.g., still or moving pictures).

In an embodiment, object tracking device 102 periodically determines andsends location information of an object 106 to camera control device104. Object tracking device 102 then determines coordinate data 116(1)of object 106(1) using location unit 112(1), and sends coordinate data116(1) via communication path 118 to camera control device 104. Cameracontrol device 104 uses the coordinate data 116(1) to determine cameraadjustments for capturing or maintaining object 106(1) within field ofview 120(1) of assigned camera 110(1). Camera control device 104 thenadjusts camera 110(1) to maintain position of object 106(1) within fieldof view 120(1). Where object 106(1) has continued motion, camera controldevice 104 may determine object speed and direction and, accordingly,control movement of camera 110(1) so that object 106(1) is trackedsmoothly (e.g., without jumping or jitter) by camera 110(1).

Camera control device 104 receives coordinate data of object 106(1) and,in an embodiment, maps the object coordinates to a reference coordinatemodel (e.g., an x, y, z three-dimensional space) to determine where topoint camera 110(1). That is, camera control device 104 determines fieldof view 120(1) for camera 110(1) based upon location of object 106(1)and controls camera 110(1) to capture that field of view 120(1). Forexample, camera control device 104 controls pan, tilt and zoom of camera110(1) to capture field of view 120(1). As object 106(1) moves, field ofview 120(1) is determined and camera 110(1) controlled to capture theupdated field of view 120(1). If coordinate data 116(1) includesvelocity data for object 106(1), camera control device 104 may calculatefield of view 120(1) accordingly and control camera movement speeds(e.g., pan speed) and/or zoom speeds to capture field of view 120(1).Camera control device 104 may determine field of view 120(1) to includeother tracked objects 106 based upon sport specific information, asdescribed below and shown in FIG. 24. For example, where object 106(1)is a soccer player and object 106(2) is a soccer ball traveling towardsthe soccer player, camera control device 104 may determine field of view120(1) to include both object 106(1) and object 106(2).

Likewise, as object 106(3) moves within operational field 108, objecttracking device 102 interacts with location unit 112(3) to periodicallydetermine and communicate coordinate data 116(3) to camera controldevice 104, via communication path 118. Camera control device 104 thencontrols camera 110(2) to capture or maintain object 106(3) within fieldof view 120(2). Where object 106(3) has continued motion, camera controldevice 104 for example controls motion (e.g., pan) of camera 110(2) forsmooth tracking of object 106(3). That is, camera control device 104controls camera 110(2) to maintain size and location of object 106(3)within a field of view 120(2) of camera 110(2) even though object 106(3)is moving. If coordinate data 116(3) includes velocity data for locationunit 112(3), camera control device 104 may use this information todetermine or update camera 110(2) movement (e.g., pan) and/or zoom sothat object 106(3) proportionally fills field of view 120(2) duringobject movement.

By including location information of cameras 110 relative to operationalfield 108 within camera control device 104, camera control device 104determines and controls each camera 110 to smoothly track assignedobjects 106 (e.g., to maintain size and location of assigned objects 106within fields of view 120). Camera control device 104 for examplecontrols pan, tilt and zoom of each camera 110 to capture object 106 ina desired position or proportion of each respective field of view 120.As object 106(1) moves away from camera 110(1), for example, cameracontrol device 104 controls pan, tilt and zoom of camera 110(1) tomaintain consistent object size throughout the image data (e.g., imageseries or stream) captured by camera 110(1). Camera control device 104may adjust the apparent size of object 106 within captured image data ofcamera 110 by zooming camera 110, for example.

In an embodiment, operational field 108 is a sports playing field andeach object 106 is a “star” player. Each player or object 106 isequipped with a location unit 112. Each camera 110 is assigned, e.g.,using user interface 114, to one player/object 106. System 100 trackseach player/object 106 assigned to a camera to maintain eachplayer/object 106 within an assigned camera's field of view 120 andconsistently record events and reactions of each player/object 106.Streamed images from these cameras are replayed (e.g., stored andre-played as on demand video streams) or recorded for later analysis(e.g., by coaching staff) or for use during televised production of thesporting event.

System 100 may provide “video replay” for officiating purposes at asporting event. If a flagrant foul is committed by a trackedplayer/object 106 “away from the action,” officials may review eachtracked player's actions to facilitate adjudication (unlike the currentsystem in American football where only the main action is recorded andreplayed). An opposing team is for example given a certain number ofopportunities to request such replay and adjudication.

System 100 may provide web-casting feeds to a web site. A user accessesthe site to select and view available images or recordings of a featuredplayer. Rather than trying to watch the featured player on a relativelylow resolution wide angle feed, as is conventionally streamed to websites, the user views recorded or substantially real-time video of thefeatured player, as captured by one or more assigned cameras. Cameracontrol device 104 controls a camera 110 assigned to the featured player(object). Camera 110 zooms in on the featured player, such that thefeatured player occupies a majority of the field of view, to provideacceptable feed of the featured player even at a lower web-castingresolution. The user may also elect to simultaneously view feed of thefeatured player and a wide-angle feed of the game in general. In anotherexample of use, a father who is unable to attend an athletic event inwhich his son is performing pays to have a camera capture images of hisson's performance and have the associated feed made available on a website. Thus, payment of a fee (e.g., camera rental) ensures cameraassignment to his son, even if the son is not a ‘star’ performerautomatically selected by system 100.

System 100 may continually image selected (i.e., tracked) players, the“area around the ball” or “the leader of a race,” to capture gamehighlights and relevant image data without requiring one or more skilledpeople at the sporting venue to operate cameras. Where the sportingvenue has multiple cameras, a user watching a web-cast of the sportingevent may select their desired view and/or player to watch, e.g., from aselection of players and views offered on the web site.

In an embodiment, coordinate data 116 includes object orientationinformation for each location unit 112. Object tracking device 102 usesthe object orientation information to determine a direction that eachtracked object 106 is facing. Camera control device 104 receives theobject orientation information via communications link 118, and controlsone or more cameras 110 to capture desired views (e.g., a front or sideview) of each tracked object 106. Camera control device 104 utilizesthis orientation information to automatically assign a camera 110 to anobject 106 that is facing the camera 110, for example. As each object106 moves about operational field 108 and the orientation informationassociated with each object 106 changes, camera control device 104 forexample reassigns cameras 110 to objects 106, to maintain desired viewsof objects 106. Accordingly, system 100 facilitates determining and thencapturing image data of the front of the moving sportsman, for example,since a moving sportsman generally faces the direction of motion andthis may be readily determined by system 100.

FIG. 28 shows an alternate system 2800 embodiment that is similar tosystem 100 of FIG. 1, but where location information is derived from twofixed cameras 2810(1) and 2810(2) that provide two overlapping imagestreams of operational field 108. Object tracking device 102 receivesthese overlapping image streams to determine location information ofeach object 106 on operational field 108. In one example, objecttracking device 102 identifies each object 106 using visibleidentification numbers (e.g., player numbers) and uniform colors fromimages of cameras 2810. Object tracking device 102 then utilizes one ormore parallax techniques to triangulate the position of each object 106within operational field 108.

FIG. 2 shows an exemplary use of a system 200 for capturing image dataof tracked objects 206 moving within a circuit 224. Objects 206 forexample represent runners, speed skaters, race cars or horses and/orjockeys, and circuit 224 represents a running track, an ice rink, a racetrack or a horse track, respectively.

System 200 is shown with an object tracking device 202, a camera controldevice 204, and ten motorized cameras 210. Cameras 210 are situatedwithin or around an operational field 208 that includes circuit 224.Circuit 224 is shown with a starting line 226. Four exemplary objects206 are shown in FIG. 2, each having a location unit 212. Objecttracking device 202 may represent object tracking device 102, FIG. 1;camera control device 204 may represent camera control device 104;operating field 208 may represent operational field 108; cameras 210 mayrepresent cameras 110; and location units 212 may represent locationunits 112.

Object tracking device 202 utilizes location units 212 to determinelocation information (e.g., coordinate data 116, FIG. 1) for objects 206within operational field 208. Object tracking device 202 sends thislocation information to camera control device 204. Camera control device204 again may include a model of operational field 208, with coordinatesof circuit 224 and each camera 210. Camera control device 204 receivescoordinate data of objects 206 from object tracking device 202,determines a possible field of view from each camera 210 to each object206, and assigns cameras 210 to the objects based upon optimum field ofview selection. For example, for each camera 210 and for each object206, camera control device 206 determines a possible field of view fromthe camera to the object. Then, by selecting an optimum field of viewfor each object (e.g., based upon the distance from the camera to theobject, the objects position within circuit 224 and whether all objectsare assigned to a camera), control device 204 determines which camera210 to assign to each object 206. Where the number of objects is lessthat the number of cameras, camera control device 204 may assign morethan one camera 210 to an object 206. Where the number of objects ismore than the number of cameras, camera control device 204 may assignone or more select cameras 210 to a more important object 206 (e.g., theleader in a race). Each object 206 is for example prioritized such thatcameras assignment is also prioritized for that object—in a race, theleader is assigned a higher priority to ensure best camera assignment.

Objects 206 proceed around circuit 224 in a direction indicated by arrow228. The direction of objects 206 is for example provided to cameracontrol device 204 with object 206 coordinate information. Cameracontrol device 204 uses the directional information for optimal cameraassignment to each object 206. As object 206(1) proceeds around circuit224, camera control device 204 for example selects and controls eachcamera 210 to capture image data of object 206(1), e.g., based upondistance between object 206(1) to each camera.

In an example of operation, when object 206(1) is located as shown inFIG. 2, camera control device 204 controls camera 210(1) to maintainobject 206(1) within the field of view of camera 210(1). As object206(1) proceeds around circuit 224 and away from camera 210(1), cameracontrol device 204 assigns camera 210(9) to object 206(1) and controlscamera 210(9) to maintain object 206(1) within the field of view ofcamera 210(9). As object 206(1) proceeds yet further around circuit 224and away from camera 210(9), camera control device 204 assigns camera210(8) to object 206(1) and controls camera 210(8) to track object206(1) within camera 210(8)'s field of view. Optionally, camera controldevice 204 selects more than one camera 210 to simultaneously trackobject 206(1) within its field of view.

Camera control device 204 may be made aware of movement characteristics(e.g., direction of movement) of an object 206, and accordingly assignsor re-assigns cameras based upon camera fields of view that include thefront of object 206. Camera control device 204 for example assumes thatthe front of object 206 faces forward as it moves, or it may instead beprogrammed to identify the front of object 206 as the aspect facing thegeneral direction of movement of an event. Camera control device 204accordingly assigns and controls cameras 210 to capture frontal and sideimage data of object 206, in preference to rear images.

System 200 also includes a recording device 220 for recording and/orconverting image data from each camera. In an embodiment, recordingdevice 220 simultaneously records image data 219 from each camera 210.Image data 219 is therefore a signal or signals representing data bitscaptured by each camera 210. Recording device 220 includes processingsoftware for converting the received signal into a data stream andinterpreting the data stream as a series of images, which are thenrecorded as video, for example. System 200 is thus suitable for use inautonomous still or moving picture production.

Camera control device 204 sends annotation data 227 to recording device220 for recording with image data 219. Annotation data 227 includesidentification of tracked objects of image data 219. For example, ifcamera 210(1) is selected to maintain object 206(1) within its field ofview, as the image data from camera 210(1) is recorded by recordingdevice 220, camera control device 204 may include identification ofobject 206(1) within annotation data 227 that is recorded with the imagedata. As shown, recording device 220 may generate (i.e., burn) a disc222 (e.g., a DVD or CD disc) representative of the recorded image data.

In one operational example, a runner (e.g., object 206) rents a locationdevice (e.g., location device 212) for a certain period while trainingat a running track (e.g., circuit 224). System 200 identifies the runnerwithin operational field 208 and object tracking device 202 sendscoordinate data of location unit 212 to camera control device 204. Sincecamera control device 204 is aware of the location and boundaries ofcircuit 224 within operational field 208 and of movement characteristicsof objects 206 (i.e., runners) performing on circuit 224 (i.e., runningtrack), camera control device 204 also determines performance (e.g., laptimes, number of laps, average speeds, etc.) of these runners (e.g.,object 206(1)) when they behave according to these characteristicswithin circuit 224. The determined performance is for example includedwithin annotation data 227 and recorded by recording device 220. As therunner 206(3) lines up and remains stationary for a certain period atstarting line 226, camera control device 204 determines that runner206(3) is about to start running laps of circuit 224 and, as runner206(3) starts moving, camera control device 204 starts a timer forrunner 206(3) and measures performance of runner 206(3). When runner206(3) completes a training session, runner 206(3) obtains (e.g.,purchases) a disc 222 that includes video and/or still images of runner206(3)'s training session and, optionally, performance information.

System 200 may operate to capture image data of objects 206 within theknown boundaries of circuit 224. That is, in an embodiment, if an object206 leaves the area of circuit 224, camera control device 204 no longerassign cameras 210 to that object, thereby no longer recording imagedata related to objects 206 external to the area of circuit 224. Asshown in FIG. 2, object 206(5) is located outside of circuit 224. Eventhough object 206(5) has an attached location unit 212(5), cameracontrol device 204 need not assign any cameras 210 to object 206(5)until object 206(5) enters circuit 224.

There may be little interest in capturing images of an athlete warmingup on the side lines. In an embodiment, circuit 224 is an Americanfootball field and camera control device 204 is configured to assignobjects 206 to cameras 210, while objects 206 are within a certain area(which may be more or less than the actual football field). However, inthis embodiment one or more cameras 210 are continually assigned toselected objects 206 while they are within operational field 208. System200 thus allows for continuous capture of popular players, even whilethey are on or off the field. Likewise, in an embodiment, one object 206is a football, and at least one camera 210 is continually assigned tothe football to capture action (e.g., errant throws or sideline tackles)outside the boundaries of the football field.

Recording device 220 may combine annotation data 227 with recorded imagedata 219 when generating disc 222. Continuing with the above runningtrack example, recording device 220 selects performance data for runner206(3) and includes this performance data, e.g., as a video overlay,when recording image streams of runner 206(3) onto disc 222. This videooverlay is for example formatted as tabulated figures that include lapand cumulative times and/or a graphical representation of runner206(3)'s performance.

In another embodiment, recording device 220 replays recorded image data219 and annotation data 227 as a single feed 205 featuring one or moreobjects 206. Where system 200 is utilized as a training device by asports team, recording device 220 may be operated to generate image feed205 by overlaying annotation data 227 onto image data 219 for one ormore selected athletes. Thus, recording device 220 automaticallydisplays recorded image streams and performance information of theselected athlete. The video overlay and performance information includedtherein is variable according to sport and/or preference or selection ofa user of recording device 220.

In one embodiment, recording device 220 delivers instant replay imagesor video streams that include overlaid performance informationdetermined from annotation data 227 for the object(s) 206 associatedwith image data 219.

In one embodiment, recording device 220 generates live image feed 205 bycombining a video overlay of performance information selected fromannotation data 227 and image data 219. In particular, recording device220 of this embodiment matches performance information from annotationdata 227 to image data 219 for each object 206 to which a camera 210 isassigned.

In one embodiment, recording device 220 copies recorded image data 219and associated annotation data 227 to disc 222 for later processing. Forexample, annotation data 227 and image data 219 of this embodiment maybe copied in a raw data format for processing and replay on a separatedevice (not shown), e.g., a computer with video conversion capabilities.

FIG. 3 shows an embodiment of a system 300 that captures images of oneor more snow athletes (e.g., snow-boarders or skiers) performing withina half-pipe 324. System 300 is similar to systems 100 and 200. System300 is shown with an object tracking device 302, a camera control device304, a plurality of cameras 310 and a recording device 320. Half-pipe324 is located within an operational field 308 of system 300.Operational field 308 is larger than half-pipe 324 such that system 300identifies each snow athlete 306 as he or she approaches half-pipe 324;it is thus ‘prepared’ to capture image data of snow athlete 306 prior toimportant action.

In one operational scenario, one or more snow athletes 306 rent locationunits 312 for a certain period (e.g., one day or one hour). During thisperiod, if a snow athlete 306 performs within half-pipe 324, system 300records image data of snow athlete 306's activities within half-pipe 324and optionally determines and records performance characteristics ofathlete 306 within half-pipe 324. Performance characteristics forexample include jump height, speed, number of rotations, number of jumpsperformed, etc. As taught by FIG. 2, identification and performance datafor athlete 306 may be recorded with image data of athlete 306 byrecording device 320. Recording device 320 may convert the image dataand any identification/performance data to digital video to generate aviewable disc (e.g., a DVD), or record the data in raw form, for laterprocessing

Once the rental period for location unit 312 is over, recording device320 generates (i.e., burns) a disc 322 (e.g., a DVD or CD) of athlete306's performance, including any annotated performance information.Recording device 320 optionally or additionally saves athlete 306'sperformance to a memory card or like digital media. Alternately oradditionally, recording device 320 includes software and programinstructions for facilitating download or transmission (even wirelesstransmission) of athlete 306's performance from recording device 320 toa portable video player, such as an Apple iPod™, a personal computer ora cell phone (e.g., via Bluetooth or another (e.g., cellular)communications link). Likewise, athlete 306's performance may be podcastand a URL provided to athlete 306, for subsequent downloading to acomputer, a portable video player or a similar video-capable device.

FIG. 4 shows one exemplary embodiment of a system 400 for capturingimages of grouped tracked objects 406. System 400 is shown with anobject tracking device 402, a camera control device 404, a plurality ofcameras 410 and a plurality of location devices 412, each attached to atracked object 406 within an operational field 408. System 400 issimilar to systems 100, 200 and 300 of FIGS. 1, 2 and 3, respectively.

Object tracking device 402 uses location units 412 to determine locationinformation for each tracked object 406 within operational field 408.This location information is sent to camera control device 404, whichassigns one or more cameras 410 to each tracked object 406, for examplebased upon the location of cameras 410 and objects 406. Camera controldevice 404 outputs image data 419 received from cameras 410 as a livefeed 405.

In an embodiment, camera control device 404 includes algorithms 426 thatare tailored to various sports, taking into account their unique objectmovement characteristics. Algorithms 426 for example includecharacteristics for one or more of objects 406 (i.e., certain players orpositions); thus, camera control device 404 is ‘aware’ of expectedbehavior of these objects within operational field 408. Where system 400is operating at a soccer game, for example, algorithms 426 may governassignment or reassignment of one or more cameras to a group of objects,to capture and/or maintain action in the vicinity of the soccer ballwithin the field of view of the camera(s). Cameras 410 are for examplecontrolled to ‘zoom in’ on an area occupied by the group of objects, andto zoom out as the group disperses. Algorithms 426 for example governcamera control device 404 operations to maintain a certain number oftracked objects 406 within the field of view of a particular camera 410.Other cameras 410 may be simultaneously controlled by camera controldevice 404 to maintain assigned objects within their fields of view.

In one example of operation, one tracked object 406 is a ball usedwithin a sport being played within operational field 408 by players 406.The ball for example has an embedded location unit 412. Since in manyball sports most of the relevant sporting action occurs in the locationof the ball, by determining the ball location and selecting anappropriate field of view, sporting action is automatically tracked byone or more cameras 410. Further, by including algorithms (e.g.,algorithms 426) that evaluate certain characteristics of the determinedball movement, sporting plays may be anticipated and captured.

FIG. 5 shows an embodiment of a system 500 with a mobile camera 510(1)for capturing image data of tracked objects 506. System 500 is shownwith an object tracking device 502, a camera control device 504, amobile camera 510(1), two fixed position cameras 510(2) and 510(3) and aplurality of location devices 512, each attached to a tracked object 506within an operational field 508. System 500 is, for example, similar tosystems 100, 200, 300 and 400 of FIGS. 1, 2, 3 and 4, respectively.Camera control device 504 receives image data 519 from each camera 510that is output as a live feed 505. System 500 may have more mobilecameras without departing from the scope hereof.

In system 500, mobile camera 510(1) is horizontally mobile and laterallymobile (e.g., mounted upon a laterally mobile platform such as a wirestrung between two structures) and its lateral position is controlled bycamera control device 504. Mobile camera 510 may also be verticallymobile.

In an example of operation, object 506(2) is assigned to camera 510(2),object 506(3) is assigned to camera 510(3) and object 506(1) is assignedto camera 510(1). Cameras 510(2) and 510(3) are controlled by cameracontrol device 504 to maintain objects 506(2) and 506(3), respectively,within their fields of view. As object 506(1) moves (as indicated byarrow 530), camera control device 504 controls camera 510(1) such thatcamera 510(1) moves (arrow 532) at a similar speed and in the directionof object 506(1) and maintains object 506(1) within the field of view ofcamera 510(1), without losing quality of imagery of object 506(1). Bycontrolling the lateral position of camera 510(1), further enhancementsin the quality of captured images may result. In the case of an Americanfootball game, where camera 510(1) is assigned to a receiver, imaging ofplays involving this receiver may benefit from the mobility of camera510(1), since camera 510(1) is automatically controllable to ‘follow’the receiver down the field.

Continuing with this American football example, by including a locationunit 512 within the ball, camera control device 504 determines andpredicts angles between the ball, the receiver and the camera to allowoptimal selection, positioning and field of view for camera 510(1) tocapture expected ‘plays.’ System 500 may thereby provide higher qualityimagery than conventional recording setups.

In another example of operation, at sports related events where largetelevision screens are provided to show live action and instant replays,systems 100, 200, 300, 400 and 500 may be used to provide both live andinstant replay images (e.g., still and/or moving pictures) to thesetelevision screens. At such events as an American Football game, aproduction team selects camera views and replay clips for display uponthese television screens. An operator watches the sporting event andprovides certain ‘event’ inputs to an image recording device, such thatan instant replay, should it be requested by a producer, may be selectedfor display. Based upon these event inputs, digitization software marksdigitally recorded ‘clips’ thereby allowing rapid selection and replayof these clips as needed. Continuing with the American football example,the operator indicates to the digitization software when the snaphappens, when the whistle blows and indicates a play type (play actionpass, play action run, roll out, etc.) that has occurred. Thedigitization software marks image data related to these events, and themarked clips are immediately available to the producer for instantreplay. Systems 100, 200, 300, 400 and 500 may also operate in thismanner, with the advantage that image data from each camera may besimultaneously marked and clipped for picture production, therebyproviding a plurality of camera angles for the instant replay. Further,since each tracked player is identified within annotation dataassociated with the image data captured for that player, the producermay identify the image clips by one or more of (a) the position (and/orname) of the player, (b) the type of event and (c) the time of theevent. A database may be used to track camera/object assignments at anygiven time, thereby facilitating recall of specific image clips.

FIG. 6 shows an embodiment of a system 600 that captures unobstructedimage data and/or recording unobstructed images of tracked objects 606.System 600 is, for example, similar to systems 100, 200, 300, 400 and500 of FIGS. 1, 2, 3, 4 and 5, respectively. In particular, system 600has an object tracking device 602, a camera control device 604, aplurality of cameras 610 and a plurality of location units 612, eachattached to one object 606 within an operational field 608. Image data619 from each camera 610 is output as a live feed 605. System 600 isalso shown with an optional production control device 614, external tocamera control device 604. In an alternate embodiment, functionality ofproduction control device 614 is included within camera control device604.

Operational field 608 is shown including two visual obstructions 632 and634; these visual obstructions 632, 634 are for example structuralobjects such as pillars within an arena, visual props, speaker stacks ona stage, referees, non-star players or other objects that obstruct anobject 606 from a camera 610. Camera control device 604 receivescoordinate data from object tracking device 602 for each location unit612 within operational field 608, is aware of the location of eachcamera 610 and is aware of the location and size of each visualobstruction 632 and 634. Thus, camera control device 604 determines ifeach object 606 (e.g., a musician, performer or athlete) is visuallyobstructed from each camera 610 by visual obstructions 632, 634 or otherobjects 606.

In particular, in the example of FIG. 6, camera control device 604 maydetermine that object 606(3) is visually obstructed from cameras 610(2),610(3) by object 606(2), and visually obstructed from camera 610(4) byobject 606(4). Camera control device 604 therefore assigns camera 610(1)to object 606(3) and maintains object 606(3) within the field of view ofcamera 610(1). Similarly, camera control device 604 may determine thatthe line of sight between camera 610(1) and object 606(1) is obscured byobstruction 632, the line of sight between camera 610(3) and object606(1) is obscured by object 606(2) and the line of sight between camera610(4) and object 606(1) is obscured by obstruction 634. Camera controldevice 604 therefore pairs object 606(1) with unobstructed camera 610(2)to capture image data 619 related to object 606(1). Camera 610(2) iscontrolled to maintain object 606(1) within its field of view. The fieldof view of camera 610(3) is similarly obscured for objects 606(1),606(3) and 606(4), and therefore camera control device 604 assignscamera 610(3) to object 606(2) and maintains object 606(2) within thefield of view of camera 610(3). Camera control device 604 then assignsobject 606(4) to camera 610(4) and maintains object 606(4) within thefield of view of camera 610(4), for example after determining thatobjects 606(1), 606(2) and 606(3) are obscured from view by camera610(4) by obstruction 634 and object 606(4).

It should be noted that since each object 606 is also tracked withinoperational field 608, camera control device 604 may determine whether afield of view of one object 606 (e.g., a star player) is blocked byanother object 606 (e.g., a referee or a non-star player). Further,since each object may move within operational field 608, camera controldevice 604 may predict if one object is about to be blocked by anotherobject or by a non-movable obstruction and select an alternate field ofview prior to the obstruction.

Camera control device 604 may continuously evaluate each possible fieldof view from each camera to determine optimum camera selection asobjects 606 move within operational field 608. Further, camera controldevice 604 may include selection hysteresis to prevent repeated and/ortoo rapid camera and/or field of view switching.

In an embodiment, production control device 614 provides user input formanual camera selection based upon indicated images and predicted fieldof view obstructions as objects 606 move within operational field 608.

In an embodiment, camera control device 604 models operational field608, cameras 610, objects 606 and obstructions 632, 634 to determiningoptimal field of view selection.

Where operational field 608 of system 600 represents a baseball fieldand stadium, the trajectory of a hit ball (determined by system 600where the ball includes a location unit 612, or determined by externalmeans and relayed to system 600) may be evaluated within camera controldevice 604 to determine where the ball will land. Camera control device604 controls one or more cameras 610 to image the predicted landinglocation of the ball.

As appreciated, systems 100, 200, 400, 500 and 600 may be utilized invirtually any sport. These systems may be used to provide features thathave never before been available to “high end” TV productionapplications. Further, these systems may enable “low end” applicationsto have access to “fully automated” sports TV production withoutrequiring skilled operators.

FIG. 7 shows a partial perspective view of a camera control system 700with exemplary control zones 732 and 734 within an operational field 708that may represent any one of operational fields 108, 208, 308, 408, 508and 608 of FIGS. 1, 2, 3, 4, 5 and 6, respectively. Operational field708 may have more or fewer control zones without departing from thescope hereof. In FIG. 7, an object 706 and an attached location device712 are located within control zone 732. Object 706 for examplerepresents a referee, and operational field 708 represents an Americanfootball field. Control zones 732 and 734 represent locations upon thefootball field where the referee stands to address a camera whendelivering adjudication of play. System 700 uses control zones 732 and734 to attach additional information to recorded image data. Continuingwith the American football example, upon detecting the referee (anobject 706) within any one of these control zones, camera control system700 may automatically include output from the referee's microphone withthe associated image data.

FIG. 8 shows a partial view of one exemplary camera control system 800,illustrating a control zone 832 within an operational field 808 thatincludes a circuit 824. Circuit 824 may represent circuit 224 of FIG. 2.In the example of FIG. 8, zone 832 represents an area in the vicinity ofa starting line 826. A location unit 812 is attached to an object 806(e.g., a runner) who is preparing to start a training session bystanding still at starting line 826 within zone 832. As previously notedwith regard to FIG. 2, the stationary nature of object 806 at startingline 826 may indicate the start of the training session. Further, asdescribed below with respect to FIGS. 9-11, control zone 832 allowsobject 806 to provide additional information to system 800.

FIGS. 9, 10 and 11 show perspective views 900, 1000 and 1100illustrating camera control traces for camera control; in particularthese views detail exemplary control zones 932, 1032 and 1132,respectively, which may represent any of control zones 732, 734 and 832,and which may be employed with any of systems 100, 200, 300, 400, 500and 600. Within these control zones, location devices (not shown) may beput through traces 934, 1034 and 1134, to command camera control devices104, 204, 304, 404, 504 and 604 (not shown).

FIGS. 7, 8, 9, 10 and 11 are best viewed together with the followingdescription. Camera control devices and location units are not shown forclarity of illustration. In an embodiment, movements of location deviceswithin zones 732, 734, 832, 932, 1032 and 1132 are analyzed by thecamera control device, and predetermined commands are implemented to thecamera control device when movements of location devices correspond tocontrol zones. For example, as shown in FIG. 9, trace 934 is a letter‘S’. Moving a location device (e.g., attached to a player, runner orreferee) through trace 934 communicates the start of a certain ‘event’or ‘action’ to a camera control device. S-shaped trace 934 for examplecommands and cues the recording device to begin recording image data ofthe object (e.g., the player, runner or referee) that puts the locationdevice through trace 934. Similarly, trace 1034 is shown as a letter‘E’. Making E-shaped trace 1034 with a location device tells the cameracontrol device to stop recording image data for the object associatedwith that location device. Trace 1134 is shown as a letter ‘Z.’ Upondetecting Z-shaped trace 1134, the camera control device may ‘zoom-in’to maximize the object associated with the Z-shaped trace within acamera's field of view. In one example of use, a sports officialattaches a location unit to one or both wrists to allow a camera controldevice to recognize gestures indicating impending announcements orjudgments.

In an embodiment, a control zone covers the entire operational field,such that a camera control device recognizes traces made by any locationdevice, anywhere on the operational field. Alternately, only traces madeby specified location units on the field are recognized by the cameracontrol device. In another example, an athlete within the operationalfield wears a ‘traceable’ location device and uses predetermined tracesto ‘mark’ image data corresponding to game events, for later viewing.

Where system 100, 200, 300, 400, 500 and/or 600 is used on a ski slopeor upon a ski race course, certain deviations from expected behavior ofeach object may cause camera control devices 104, 204, 304, 404, 504 and604 to generate an alert, such as to indicate a fallen skier (i.e., theskier does not move for a certain period, which may be recognized by thecamera control device as indicating a need for assistance). In anembodiment, location units are provided for each skier (e.g., built intothe ski pass) and a camera control device (104-604) detects and recordsundesirable behavior, such as excessive speed, with one or more cameras,thereby providing evidence for disciplinary measures. In addition,cameras may be located at strategic locations on a slope or within aterrain park. A skier or snowboarder that wishes to be recorded withinsuch a strategic location checks out a location unit and (within acontrol zone associated with the location) makes a trace (e.g., the ‘S’trace of FIG. 9) to initiate a ‘recording session.’ The individual isidentifiable based upon identifying data of his or her location unit (asdescribed above with respect to FIGS. 1 and 2), to facilitate deliveryof recorded image data, pictures, video or video streams. DVDs or CDs ofthe individual's recording session are for example generated, or a URLlinking to the recorded session are provided, upon payment for achecked-out location unit.

In an embodiment, where systems 100, 200, 300, 400, 500, 600 and 700provide a live feed to a television network, if a short delay isincorporated in the feed (as occurs with the 1^(st) and 10 yellow linein televised football), camera selection is automatically made withoutthe risk of ‘choppy’ results. Each camera control device (or productioncontrol device) includes one or more algorithms to determine which imagestream from multiple cameras to select for the best quality feed.Quality of feed is maintained by predicting and editing images orswapping fields of view where the targeted object becomes obscured byother objects within the operational field. However, if the field ofview of the target object is only obscured for a few video frames, itwould be distracting to swap to another field of view and then backagain. A delay incorporated into the live feed facilitatesidentification of such swapping events before any undesirable field ofview swap occurs. Similarly, algorithms may be employed to track andpredict object movement, to reduce capture of image data while a targetobject is obscured from a camera's field of view.

Where the image feed is processed for later replay (i.e., not live), asingle composite video feed may be created using a buffer window ofarbitrary size that allows automatic selection of image stream basedupon annotation data and determination of obscured target objects. Thisassumes that the location information is included within the annotationdata that is stored with the image stream.

FIG. 12 shows one exemplary process 1200 for capturing image data oftracked objects; process 1200 may be performed autonomously (andautomatically) to produce image data that is for example suitable forstill and/or moving picture production. Process 1200 is, for example,implemented by one or more of location units 112, cameras 110, objecttracking device 102 and camera control device 104 of FIG. 1 (that is,the processing intelligence that implements process 1200 may beimplemented by one of these devices (e.g., typically the camera controldevice) or shared among different devices since communication existstherebetween). In step 1202, process 1200 assigns each of one or morecameras to the tracked objects. In one example of step 1202, camera110(1) of FIG. 1 is assigned to object 106(1) and camera 110(2) isassigned to object 106(3). In another example of step 1202, a userinteracts with user interface 114 to assign object 106(1) to camera110(1) and to assign object 106(3) to camera 110(2). In step 1204,process 1200 determines the location of each tracked object within anoperational field. In one example of step 1204, object tracking device102 interacts with location units 112(1), 112(2), 112(3) and 112(4)within operational field 108 to determine location of objects 106(1),106(2), 106(3) and 106(4), respectively, and then sends coordinate data116(1), 116(2), 116(3) and 116(4) of objects 106(1), 106(2), 106(3) and106(4), respectively, to camera control device 104. In step 1206,process 1200 determines, for each of the one or more cameras, a field ofview from the location of the camera to the determined location of theobject assigned to the camera. In one example of step 1206, cameracontrol device 104 determines field of view 120(1) for camera 110(1),such that field of view 120(1) includes object 106(1), and field of view120(2) for camera 110(2) such that field of view 120(2) includes object106(3). In step 1208, process 1200 commands each camera to capture thedetermined field of view. In one example of step 1208, camera controldevice 104 controls cameras 110(1) and 110(2) to capture fields of view120(1) and 120(2), respectively. Steps 1204 through 1208 repeat suchthat as objects (e.g., objects 106) move within the operational field(e.g., operational field 108), cameras (e.g., cameras 110) arecontrolled (e.g., by camera control device 104) to maintain the assignedobjects within their fields of view.

FIG. 13 is a flowchart illustrating one exemplary process 1300 forcapturing unobstructed images of tracked objects; process 1300 may beperformed autonomously (and automatically) such that images are forexample suitable for use in still and moving picture production. Process1300 is, for example, implemented by one or more of location units 212,cameras 210, object tracking device 202 and camera control device 204 ofFIG. 2. In step 1302, process 1300 determines location of each of thetracked objects within an operational field, using location unitsattached to the tracked objects. In one example of step 1302, objecttracking device 202 determines location information for each of objects206 within operational field 208, using location units 212, and sendsthis location information to camera control device 204. In step 1304,process 1300 determines, for each of one or more cameras, a possiblefield of view of the camera to the determined location of each of thetracked objects. In one example of step 1306, camera control device 204determines a possible field of view of each camera 210 to include eachobject 206 based upon the location of cameras 210 and the location ofobjects 206 determined by object tracking device 202. In step 1306,process 1300 determines the optimum field of view for each trackedobject. In one example of step 1306, camera control device 204 evaluateseach possible field of view of each camera 210 of each object 206 todetermine an optimal field of view selection for each camera 210. Instep 1308, process 1300 selects, for each of the one or more cameras,one of the possible fields of view of the camera based upon thedetermined optimum fields of view. In one example of step 1308, cameracontrol device 204 selects, for each of cameras 210, one possible fieldof view of the camera based upon optimum fields of view determined foreach object 206. In step 1310, process 1300 controls each of the one ormore cameras to have the selected field of view to capture an imagestream of the associated object. In one example of step 1310, cameracontrol device 204 controls each of cameras 210 to capture image streamsof at least one object 206 based upon the field of view selected for thecamera.

Steps 1302 through 1310 are repeated periodically such that the trackedobjects (e.g., objects 206) are maintained within the field of view ofone or more cameras (e.g., cameras 210).

FIG. 14 shows a system 1400 for capturing image streams 1419 of trackedobjects 1406. System 1400 is, for example, similar to systems 100, 200,300, 400, 500 and 600 of FIGS. 1, 2, 3, 4, 5 and 6, respectively. Inparticular, system 1400 has an object tracking device 1402, a cameracontrol device 1404, a plurality of cameras 1410 and a plurality oflocation units 1412, each attached to one object 1406 within anoperational field 1408. In one embodiment, image streams 1419 from eachcamera 1410 are input into image stream buffers 1416 that delay eachimage stream 1419 prior to its arrival at a production control device1414. Production control device 1414 selects and switches betweendelayed image streams 1419 to produce a live feed 1405. In an alternateembodiment, functionality of production control device 1414 is includedwithin camera control device 1404.

Camera control device 1404 receives annotation data 1427 (that includescoordinate data) from object tracking device 1402 for each location unit1412 within operational field 1408 and is aware of the location of eachcamera 1410. Thus, camera control device 1404 determines and assigns oneor more of objects 1406 to each camera 1410.

Camera control device 1404 may for example use any one or more of thepreviously disclosed methods and algorithms to assign camera 1410(1) toobject 1406(3) and maintain object 1406(3) within the field of view ofcamera 1410(1). Similarly, camera control device 1404 may pair: object1406(1) with camera 1410(2) to capture image data related to object1406(1); camera 1410(3) with object 1406(2) to capture of image datarelated to object 1406(2), and object 1406(4) with camera 1410(4) tocapture of image data related to object 1406(4).

In one embodiment, camera control device 1404 continuously evaluateseach possible field of view from each camera to determine optimum cameraselection as objects 1406 move within operational field 1408. Further,camera control device 1404 includes selection hysteresis to preventrepeated and/or too rapid camera and/or field of view switching.

Object tracking device 1402 receives annotation data 1427(1) fromlocation unit 1412(1); annotation data 1427(2) from location unit1412(2); annotation data 1427(3) from location unit 1412(3), andannotation data 1427(4) from location unit 1412(4). Object trackingdevice 1402 sends annotation data 1427 to camera control device 1404which in turn sends it to production control device 1414. In addition tocoordinate data, annotation data 1427 may include sensed or relayed dataindicating conditions or happenings related to or affecting object 1406(such as weather information received from a remote weather station) andbiometric information (e.g., heart rate, respiratory rate, etc.). InFIG. 14, a sensor module 1413 is shown collocated with location unit1412 to sense biometric information of object 1406 and transmit thatinformation to object tracking device 1402. Sensor 1413 may beconfigured with location device 1412 or may operate independently fromlocation unit 1412. In one embodiment, production control device 1414includes (or uses) a recording device 1418 for recording each imagestream 1419 together with its associated annotation data. In anotherembodiment, camera control device 1404 utilizes biometric informationwithin annotation data 1427 to determine optimal camera assignment.

Optimal camera assignment may be determined by identifying events ofinterest from the biometric information within annotation data 1427. Forexample, a sudden increase in an athlete's heart rate may indicate anevent of interest; detection of a competitor catching “big air” mayindicate an event of interest; a skier losing control may indicate anevent of interest, and a racing car becoming inverted may indicate anevent of interest. By identifying certain characteristics withinreceived annotation data 1427, camera control device 1404 and/orproduction control device 1414 are notified of events of interest withinoperational field 1408. Production control device 1414 may thusdetermine which image streams 1419 to send to recording device 1418 orto output as live feed 1405.

As noted above, annotation data 1427 may also include informationrelated to external conditions, such as weather, that affect an object1406. Object tracking device 1402 may, for example, receive weatherreports from a remote weather station during a yachting competition, andrecord the weather information with image streams 1419 of boatsparticipating in the competition. Where operational field 1408 is large(e.g., a wide geographical, as used in yachting), weather informationmay be considered an event of interest. Production control device 1414may determine which image streams 1419 capture the event of interest,and may thus send these image streams 1419 to recording device 1418 oroutput these streams as live feed 1405. Optionally, if weatherconditions such as heavy rains would prevent capture of clear images ata particular location, production control device 1414 may filter outimages streams from the particular location, and record/output imagescaptured by cameras at other locations. Optionally or additionally,cameras 1410 may be controlled to capture image streams from thelocations of interest and/or in response to an event of interest. Forexample, object 1406 may be a receiver wearing an object tracking device1402. A pass from a quarterback (also wearing a location device) to thereceiver may constitute an event of interest that triggers capture ofimages or image streams 1419 of the quarterback and the receiver, in anattempt to capture a catch by the receiver. The football passed betweenthe quarterback and the receiver may likewise have a location unit, suchthat trajectory of the ball from the quarterback in the direction of thereceiver may be determined and cameras 1410 controlled to capture thepredicted catch by the receiver.

In an embodiment, annotation data 1427 includes information pertainingto snow conditions at various locations on a cross-country ski course.Annotation data 1427 may thus be used to predict locations ofinterest—those areas where events of interest (e.g., falls orparticularly fast paces) might occur. Camera control device 1404 and/orproduction control device 1414 are notified of predicted locations ofinterest within operational field 1408. Production control device 1414may thus determine which image streams 1419 to send to recording device1418, or to output as live feed 1405.

In particular, within production control device 1414, annotation data1427 is received prior to any associated image streams, since each imagestream 1419 is delayed by image stream buffer 1416. Thus, upon detectionof an event of interest within annotation data 1427(3), for example,production control device 1414 transitions to output delayed imagestream 1416(3) as live feed 1405. In one example, where productioncontrol device 1414 is used for post-processing of recorded imagestreams 1419, annotation data 1427 associated with each image stream isprocessed to identify and alert a producer (i.e., a user of productioncontrol device 1414) to events of interest, thereby facilitating thepost-production process. This annotation data may be encoded visuallywithin the associated image stream to enhance viewer experience.

In one embodiment, system 1400 utilizes annotation data 1427 todetermine events of interest recorded (e.g., on recording device 1418)throughout a defined period (e.g., one day). These events may beidentified and output as a slightly delayed live feed 1405, or recordedfor later display. For example, system 1400 may record athletesperforming on a slalom course or within a half-pipe at a terrain park.Then, at the end of the day, system 1400 displays the ten mostsignificant recorded events (such as the top ten runs of the day, or thetop ten “big air” moments of the day) within a bar area of a lodgeassociated with the terrain park. These events are for exampleautomatically identified and compiled into a recording by productioncontrol device 1414 and recording device 1418. In another example,system 1400 is used to record an American football match and allow postgame display of events identified by processing of associated annotationdata. In particular, a coach may define certain parameters within theannotation data that are of interest, thereby allowing identification ofthese events and easy retrieval of associated image streams.

Biometric and movement data of objects 1406 in separate playing fields1408 may likewise be captured by cameras 1410 and object trackingdevices 1402, and output as live feed 1405 or recorded, e.g., to a CD orDVD, by recording device 1418. In one example, system 1400 captures anindoor rowing competition. Production control device 1414 for examplereceives image streams 1419 and annotation data 1427 and outputs livefeed of objects 1406, in this case, rowers, superimposed upon a virtualwater course. Live feed 1405 is for example displayed to the rowers on atelevision screen, so that the rowers can gauge their performance versusthat of their competitors. In another example, objects 1406 are collegerunners running on playing fields 1408, in this case, standard tracks,in two separate arenas. Runners 1406 line up at starting lines at eachtrack 1408 (e.g., as discussed in connection with FIGS. 2 and 8), andstart times are synchronized between the playing fields 1408. Cameras1410, disposed at each track 1408, capture the runners' 1406performance. Production control device 1414 (which may be locatedproximate a playing field 1408, or remote from the playing fields)receives image streams 1419 and annotation data 1427 and outputs arecording or live feed, for example delayed by buffers 1416, of avirtual competition between runners 1406 at the separate arenas.

FIG. 15 shows a system 1500 for capturing high resolution images 1519 oftracked objects 1506, which again are suitable for still and/or movingpicture production. Hereafter, images 1519 may be interchangeablyreferred to as image streams 1519. System 1500 is, for example, similarto systems 100, 200, 300, 400, 500, 600 and 1400 of FIGS. 1, 2, 3, 4, 5,6 and 14, respectively. In particular, system 1500 has an objecttracking device 1502, a camera control device 1504, a plurality ofcameras 1510 and a plurality of location units 1512, each attached to anobject 1506 within an operational field 1508. Multiple location units1512 may also be attached to a single object 1506. Object 1506(4) is forexample shown with location units 1512(4) and 1512(5). It will beappreciated that additional location units 1512 may be used according totracking preferences and as permitted by size considerations. In oneexample, object 1506(4) is a soccer player wearing a location unit1512(4) near his or her foot and a location unit 1512(5) at his or hertorso. Camera control device 1504 may control cameras 1510 to captureimages 1519 featuring a location unit of interest, according to userinput, as described with respect to system 100 and method 1200, above. Auser wishing to record a high-resolution image focusing upon player1506(4)'s foot as it contacts the ball for example assigns one or morecameras 1510 to location unit 1512(4), whereas a user wishing to recorda high-resolution image focusing generally upon the body of player1506(4) may assign one or more cameras 1510 to location unit 1512(5).Assignments may be made via a user interface (not shown, see, e.g., userinterface 114; FIG. 2) communicatively coupled with camera controldevice 1504.

Images/image streams 1519 from each camera 1510 are input into adatabase control device 1515. Database control device 1515 attaches tagsto individual images 1519 using pre-defined event information as well asinformation from the camera control device 1504. Pre-defined eventinformation for example includes information identifying an event, suchas performer name(s), team names (e.g., Broncos vs. Vikings), eventname, event date, and/or the like. In an alternate embodiment,functionality of database control device 1515 is included within cameracontrol device 1504.

Camera control device 1504 receives annotation data 1527 (includingcoordinate data and/or a velocity component) as described above withrespect to FIGS. 2 and 14. Camera control device 1504 determines when itis optimal to take an image, or a series of images by analyzing thefield of view 1520 of camera 1510. In an embodiment, camera controldevice 1504 estimates motion of objects 1506 using the annotation data1527 to extrapolate positions of objects 1506 within operational field1508, to predict fields of view and camera movements that appropriatelymaintain objects 1506 within the fields of view.

Camera control device 1504 for example uses any one or more of thepreviously disclosed methods and algorithms to assign each camera 1510to an object 1506. Additionally, camera control device 1504 is aware ofthe spatial orientation of object 1506 (e.g., which direction aparticular object 1506 is facing) within operation field 1508. In anembodiment, the spatial orientation of object 1506 is determined usingmultiple location units 1512. Returning to the example of a soccerplayer 1506(4), location unit 1512(4) is for example placed at player1506(4)'s heel or at the back of player 1506(4)'s ankle, while locationunit 1512(5) is placed at player 1506(4)'s chest. Camera control device1504 may thus determine which direction player 1506(4) is facing basedon the orientation of devices 1512(4) and 1512(5). The spatialorientation of player 1506(4) may likewise be estimated based uponplayer 1506(4)'s motion. For example, camera control device 1504 may beprogrammed to recognize the direction in which player 1506(4) is movingas the “front” of player 1504(6). See description of FIGS. 1 and 2,above.

Camera control device 1504 may also be aware of the current time,location and year (e.g., by internal devices or via communications withdate and time information), and therefore be able to determine theposition of the sun. Camera control device 1504 may use the sun positionin determining the optimal time, position and angle for high resolutionimages to be taken.

In an embodiment, image streams 1519 from each camera 1510 are inputinto database control device 1515. Each image stream 1519 containsimages that are tagged with a timestamp by camera 1510. Database controldevice 1515 may attach additional tags to individual images usingpre-defined event information and annotation data 1527 provided bycamera control device 1504. Additional tags include, but are not limitedto:

-   -   information about the object in the image, such as name, player        number or position;    -   information about the location unit or units associated with the        imaged subject, such as unit number, name of a person renting        the unit and other renter information (e.g., email address);    -   the time the image was taken;    -   the name of the competition or event at which the image was        taken;    -   events associated with the image (e.g., scoring, breaking of a        record, etc.);        In an alternative embodiment, camera control device 1504        provides tagging information to camera 1510. Camera 1510 in turn        attaches one or multiple tags to every image prior to        transmission to database control device 1515, via image stream        1519.

Database control device 1515 determines the object in the image, forexample by matching the timestamp on an image in the image stream 1519with information provided by camera control device 1504 and annotationdata 1527. Annotation data 1527 may include any one of the previouslydisclosed data types (for example, data described with respect toannotation data 227, 1427). In an embodiment, camera control device 1504directs each camera 1510 to a particular object 1506. Camera controldevice 1504 provides database control device 1515 with information foridentifying the object 1506 in a high-resolution image obtained by acamera 1510. Camera control device 1504 may also provide databasecontrol device 1515 with additional tagging information to be attachedto each image.

Database control device 1515 receives images from image stream 1519 andattaches tags to each. The “pre-filtered,” tagged images are thentransmitted to an image database 1518, which uses the tags to organizeand populate the database. In an embodiment, images are sorted via humanintervention or software prior to storage in image database 1518. Asoftware program or human editor for example removes unwanted imagesprior to a deposit into database 1518. Additionally, database controldevice 1515 may add tags to images based on events occurring after theimages have been taken. Post-image events such as a change in the scoreof a game or the breaking of a record may result in an image or seriesof images taken prior to the post-image event being tagged withinformation specific to the post-image event. In one example, imagescaptured immediately prior to a field goal are tagged with dataindicating the touchdown, to facilitate identification of imagescapturing a scoring kick.

FIG. 16 shows one exemplary process 1600 for storing image data oftracked objects in a self-organizing database. An image is received, instep 1602. In one example of step 1602, image streams 1519 from cameras1510 deliver high resolution images to database control device 1515. Thereceived images may already include a timestamp tag.

The image is identified and tags are attached, in step 1604. In oneexample of step 1604, database control device 1515 uses annotation data1527 and information provided by camera control device 1504 to determineinformation about the image, such as: the player(s) in the image, thetime the image was taken, the name of the competition, the score at thetime the image was taken, and a particular event that may have occurredduring the taking of the image (e.g., a touchdown run in an Americanfootball game). In step 1606, the image is categorized in image database1518 based on the attached tags. In one example of step 1606, imagedatabase 1518 automatically creates categories and sub-categories forthe tags attached to an image. Tags may be categorized by player, bylocation unit (e.g., when location units 1512 are attached to or rentedby/for specific individuals) and/or by competition. Sub-categories mayinclude date, time, current score, or a specific event occurring afterthe image was taken (e.g., a touchdown run following a catch captured bya camera 1510). In step 1608, the image is stored according to thecategorization and sub-categorization. In one example of step 1608, theimage is stored in image database 1618 and references to the image arestored in each category and sub-category.

FIG. 17 shows a system 1700 for capturing high resolution images andimage streams 1719 of tracked objects 1706; such image streams are forexample useful in autonomous still and/or moving picture production.System 1700 is, for example, similar to systems 100, 200, 300, 400, 500,600, 1400, and 1500 of FIGS. 1, 2, 3, 4, 5, 6, 14 and 15, respectively.In particular, system 1700 has an object tracking device 1702, a cameracontrol device 1704, a plurality of cameras 1710 and a plurality oflocation units 1712, each attached to object 1706 within an operationalfield 1708.

In one embodiment, image resolution and frame rate are modified in orderto capture high resolution still images within the image stream. Forexample, using the above-described methods, camera control device 1704determines when a tracked object 1706 is facing a camera 1710, andadjusts image resolution and/or frame rate of camera 1710 to capture astill image of the front of the object (e.g., a player's face). Imageresolution is for example reduced when the object 1706 is not facing theassigned camera 1710, when blocking by another object is predictedand/or when object 1706 moves away from a location of interest (see,e.g., description of FIG. 14, above).

Image streams 1719 from each camera 1710 may be input into image streambuffers 1716 that delay each image stream 1719 prior to its arrival atproduction control device 1714 and database control device 1715.Production control device 1714 may down-sample high resolution imageswithin image streams 1719 and then select and switch between delayeddown-sampled image streams to produce a live feed 1705. Database controldevice 1715 attaches tags to individual high-resolution imagesinterspersed within the image streams 1719, using pre-defined eventinformation as well as information from the camera control device 1504.In an alternate embodiment, functionality of production control device1714 and database control device 1715 are included within camera controldevice 1704. In another alternate embodiment, functionality of thedatabase control device 1715 is included in production control device1714.

Camera control device 1704 may use any one or more of the previouslydisclosed methods and algorithms to assign each camera 1710 to an object1706. Additionally, camera control device 1704 may use any one or moreof the previously disclosed methods and algorithms to extrapolatepositions of objects 1706 within operational field 1708, and predictfields of view and camera movements that appropriately maintain objects1706 within the fields of view.

Camera control device 1704 receives annotation data 1727 (includingcoordinate data and/or a velocity component). Camera control device 1704increases frame rate and/or resolution of cameras 1710 if there is goodprobability that an unobstructed, high quality photo would be desirable.For example, camera control device 1704 determines when it is optimal totake a high resolution still image or series of images of object1706(1), and accordingly controls camera 1710(2) to increase one or moreof the frame rate and resolution, to ensure that an occurring orpredicted event is captured. Camera control device 1704 dynamicallyadapts camera resolution and may increase frame rate for short periodsof time to provide a burst mode. In an example of “burst mode,” cameracontrol device 1704 directs camera 1710, which normally produces videoat a 640×480 pixel resolution, to increase the resolution to 2560×1920.

Image streams 1719 from each camera 1710 may be input into image streambuffers 1716 that delay each image stream 1719 prior to arrival atproduction control device 1714 and database control device 1715. In anembodiment, production control device 1714 down-samples high resolutionimages within image streams 1719 and discards “extra” frames that exceeda broadcast frame rate. Production control device 1714 then selects andswitches between delayed, down-sampled image streams to produce livefeed 1705.

Database control device 1714 tags individual high-resolution imagesinterspersed within image streams 1719. Tags may be created usingpre-defined event information as well as information from camera controldevice 1704 and/or annotation data 1727, as previously described.Additional event information may be included in tags via an externalnotification device 1730. Notification device 1730 provides for examplean input to database control device 1715 that facilitates tagging ofimages based on triggers that occur after the images have been taken.Notification device 1730 may connect or wirelessly communicate withdatabase control device 1715.

In one example, notification device 1730 is formed as a wirelesstransmitter in communication with an electronic scoreboard. A wirelessreceiver is configured with or in communication with database controldevice 1715. Responsive to a change in posted score, notification device1730 emits a signal that is received and communicated to databasecontrol device 1715. Database control device 1715 then tags imagesrecorded within a pre-selected time frame prior to the signal fromnotification device 1730. In another example, notification device 1730is a user interface whereby an observer may signal an event by pressinga button. Alternately, notification device 1730 may include a pitchrecognition unit that signals database control device 1715, or a relatedreceiver, upon identifying raised register, increased cadence orincreased volume of a sportscaster providing live event commentary.Database control device 1715 tags high resolution images responsive toinput from notification device 1730 and transfers the individualhigh-resolution images to image database 1718.

In an embodiment, functionality of production control device 1714 anddatabase control device 1715 are included within camera control device1704. In another embodiment, functionality of the database controldevice 1715 is included within production control device 1714. Otherconfigurations are within the scope of this disclosure.

In another embodiment, cameras 1710 produce only high-resolution imagestreams 1719 (e.g., 2560×1920 resolution). Image streams 1719 from eachcamera 1710 are input into image stream buffers 1716 that delay eachimage stream 1719 prior to its arrival at production control device 1714and database control device 1715. Production control device 1714down-samples high resolution image streams 1719 (e.g., to 640×480resolution), and selects and switches between the delayed down-sampledimage streams to produce live feed 1705. Database control device 1715attaches tags to high resolution images in image stream buffers 1716when notification device 1730 indicates that an important event hasoccurred.

In an embodiment, system 1700 includes a global clock 1780 that providesa synchronized time between components and devices of system 1700. Forexample, cameras 1710 may time stamp one or more frames of image streams1719 such that production control device 1714 and/or database controldevice 1715 may correlate received notifications from notificationdevice 1730, annotation data 1727, and images of image stream buffers1716.

FIG. 18 shows a notification “E” received from notification device 1730,for example upon a score change, a key play, periodically, etc. However,notification E may occur a certain period after image stream capture ofthe associated action of interest, shown as time “C”. For example, afield goal kick may occur several second before the notification isreceived from notification device 1730 (i.e., updating of a scoreboardoccurs after the ball is kicked). Thus, by utilizing image stream buffer1716 to buffer a period ‘n’ of image streams 1719 (where period ‘n’ isgreater than the delay between capture of the action C and the receiptof notification E), the image stream arriving at production controldevice 1714 and/or database control device 1715 contains the associatedaction imagery for processing in association with notification E. Thus,upon receipt of notification E, database control device 1715 may extractimages from image stream buffer 1716, attach tags to each image, andstore the images in image database 1718, in association with action Cand/or notification E.

In one embodiment, as shown in FIG. 19, a replay device 1850 replays animage stream 1852 from an image database 1818 for display on a displaydevice 1820. Replayed image stream 1852 is viewed and controlled by auser 1830. Where production control device 1714 has stored highresolution still images within image database 1818 that are associatedwith the currently displayed scenes of image stream 1852, a still imageindicator 1854 is displayed on display device 1820. Viewer 1830 mayselect (e.g., click on) indicator 1854 to pause image stream 1852 andview one or more still images 1858 within a film strip 1856 on display1820. In one example of operation, when viewer 1830 selects indicator1854, image stream 1852 is paused and replay device 1850 selects stillimages 1858 associated with displayed image stream 1852 from imagedatabase 1818 and sends these images to display device 1820 via internet1840. Viewer 1830 may select (e.g., click on) one of the still images1858 to view a larger image of the selected still image 1858. Displaydevice 1820 may be a television or computer capable of displaying imagestream 1852, indicator 1854 and film strip 1856.

FIG. 20 shows a schematic diagram 2000 of a camera 2010 controlled by acamera control device 2004. Camera control device 2004 may representcamera control devices 104, 204, 304, 404, 504, 604, 1404, 1504 and 1704of FIGS. 1, 2, 3, 4, 5, 6, 14, 15 and 17, respectively. Camera 2010 mayrepresent cameras 110, 210, 210, 410, 510, 610, 1410, 1510 or 1710 ofFIGS. 1, 2, 3, 4, 5, 6, 14, 15 and 17, respectively.

Camera 2010 has an imager 2052 that represents optical and electroniccomponents for capturing an image stream 2072 under control of cameracontrol device 2004. Imager 2052 operates at a variable resolution(e.g., between a high resolution such as 2048×1538 pixels and a lowresolution such as 512×384 pixels) and with a variable frame rate (e.g.,between 1 and 1000 frames per second); both resolution and frame rateare controlled by camera control device 2004.

Camera 2010 is shown with three image data paths, herein called ‘live’,‘slow-motion’, and ‘still image’, each of which is controlled (e.g.,turned on and off) by camera control device 2004, for example accordingto the nature of the event being imaged. Operation of camera 2010 mayfollow one of eight states, shown in Table 1.

TABLE 1 Camera States State Live Slow-Motion Still Image 1 Off Off Off 2Off Off On 3 Off On Off 4 Off On On 5 On Off Off 6 On Off On 7 On On Off8 On On On

Camera control device 2004 selects the frame rate and resolution ofimager 2052 based upon received notifications 2030 that indicate anevent of interest to be captured. Camera control device operates imager2052 at a standard resolution (e.g., a resolution suitable for atelevision feed) and at a standard frame rate (e.g., thirty frames persecond) in the absence of notification 2030. Then, depending upon thetype of received notification 2030, camera control device 2004 maycontrol imager 2052 to operate at a higher resolution (e.g., 2048×1538pixels) and/or a higher frame rate (e.g., one-hundred and twenty framesper second).

Based upon notification 2030, camera control device 2004 controls imager2052 to operate at a higher resolution (e.g., 2048×1538 pixels) whenstill images are to be captured. A rate down-sampler 2054 reduces theframe rate of image stream 2072 while maintaining the high resolution ofeach remaining frame, and feeds this low frame rate (e.g., five framesper second) high resolution image stream into a still image buffer 2056.Images stored within still image buffer 2056 may then be output as stillimage feed 2066 under control of camera control device 2004. Theseoutput images may then be stored within a database (e.g., image database1718, FIG. 17) with annotation information.

Based upon notification 2030, camera control device 2004 controls imager2052 to operate at a high frame rate (e.g., one-hundred and twentyframes per second) when a slow-motion image stream is to be captured.Where resolution of image stream 2072 is higher than required for theslow-motion image stream, a resolution down-sampler 2058 reduces theresolution of each frame of image stream 2072 while maintaining theframe rate to produce a high frame rate lower resolution (e.g., 640×480pixels) image stream that is fed into a slow-motion buffer 2060 fromwhere it is output, under control of camera control device 2004, as aslow-motion feed 2068. Slow-motion feed 2066 may be used to produce aslow-motion effect when displayed at a frame rate less than the captureframe rate. For example, if slow-motion feed 2068 is captured atone-hundred and twenty frames per second and displayed at thirty framesper second, the effect is one quarter speed slow-motion.

Based upon notification 2030, camera control device 2004 may controlimager 2052 to capture image stream 2072 with increased resolution andframe rate to capture high resolution still images and a slow-motionimage feed. Down samplers 2054 and 2058 operate to reduce the frame rateand resolution of image stream 2070 to provide the desired still imagesand slow-motion feed, respectively.

A rate/resolution down-sampler 2062 operates to produce a desiredresolution and frame rate of live feed 2070, irrespective of resolutionand frame rate of image stream 2072. Camera 2010 outputs live feed 2070while optionally and simultaneously capturing high quality still imagesand/or a slow-motion image stream under control of camera control device2004.

Still image feed 2066, slow-motion feed 2068 and live feed 2070 may becombined into a single digital feed 2064, without departing from thescope hereof. Each feed may be tagged with the captured frame rage andresolution to facilitate later processing. Camera 2010 is particularlysuited for use where notification 2030 indicate events of interest thatare yet to happen, thereby allowing camera control device 2004 tocontrol camera 2010 to capture still images and slow-motion feedappropriately for the event. Further, by controlling imager 2052 tocapture image stream 2072 with an appropriate resolution and frame ratefor still images and slow-motion only as necessary, imager 2052 operatesto capture image stream 2072 at optimal quality. That is, imager 2052does not operate at a frame rate and resolution higher than required byany one of desired output feeds 2066, 2068 and 2070. For example,capturing an image stream at a higher frame rate than necessary mayresult in poorer image quality. Similarly, capturing an image stream ata higher resolution and then down sampling may also result in poorerimage quality.

Shutter speed of imager 2052 may also be controlled by camera controldevice 2004 based upon the type of event being captured. For example,camera control device 2004 may ensure a fast shutter speed (e.g.,1/250^(th) of a second or smaller) is used by imager 2052 when capturingimages of a sporting event to reduce image blur resulting from fastmoving subjects.

Although shown with still image, slow-motion and standard image streamcapture capability, camera 2010 may be configured to include acombination of these functionalities without departing from the scopehereof. For example, rate down-sampler 2054 and still image buffer 2056may be omitted when still image capture functionality is not required;resolution down-sampler 2058 and slow-motion buffer 2060 may be omittedwhen slow-motion capture functionality is not required; andrate/resolution down-sampler 2062 may be omitted when standard feed 2070is not required. Further, camera 2010 may have additional image datapaths, such as to include two slow-motion paths, each operating atdifferent frame rates, without departing from the scope hereof.

In one embodiment, camera 2010 has a limited bandwidth and is notcapable of running at its maximum resolution and maximum frame ratesimultaneously. Camera control device 2004 thus controls imager 2052 tooperate within constraints imposed by camera 2010.

FIG. 21 shows a schematic diagram 2100 of a camera 2110 that continuallycaptures still images and a slow-motion image stream and is controlledby a camera control device 2104. Camera control device 2104 mayrepresent camera control devices 104, 204, 304, 404, 504, 604, 1404,1504 or 1704 of FIGS. 1, 2, 3, 4, 5, 6, 14, 15 and 17, respectively.Camera 2110 may represent cameras 110, 210, 210, 410, 510, 610, 1410,1510 or 1710 of FIGS. 1, 2, 3, 4, 5, 6, 14, 15 and 17, respectively.

Camera 2110 is similar to camera 2010, FIG. 20, and further includescircular buffers 2155, 2159 to continually store still images and aslow-motion image stream. An imaging unit 2152 of camera 2110 representsoptical and electronic components for capturing a high resolution (e.g.,2048×1538 pixels) fast frame rate (e.g., between 30 and 1000 frames persecond) image stream 2172. A rate down-sampler 2154 reduces the framerate of image stream 2172 while maintaining the high resolution, andfeeds this high resolution low frame rate (e.g., five frames per second)image stream into circular buffer 2155. Circular buffer 2155 thus storesstill images for a period, depending upon the size of circular buffer2155, up to present. When camera control device 2104 receivesnotifications 2130 indicating an event of interest, camera controldevice 2104 may transfer still images from circular buffer 2155 to stillimage buffer 2156 for a period surrounding the time of the event ofinterest, provided that the time of the event of interest falls withinthe buffered period of circular buffer 2155. These kept images may thenbe output as still image feed 2166.

A resolution down-sampler 2158 reduces the resolution of each frame ofimage stream 2172 while maintaining the high frame rate to produce alower resolution high frame rate image stream that is fed into acircular buffer 2159. Thus, circular buffer 2159 stores, depending uponthe size of circular buffer 2159, a period of slow-motion image streamup to present. When camera control device 2104 receives notifications2130 indicating an event of interest, camera control device 2104 maytransfer a slow-motion image sequence that represents a periodassociated with the time of the event of interest to a slow-motionbuffer 2160 from where they are output as a slow-motion feed 2168.

In one embodiment, circular buffers 2155 and 2159 are sized to storesixty seconds of still images and slow-motion image stream,respectively, thereby allowing still images and slow-motion imagestreams to be provided even when notifications 2130 identify events ofinterest that have already occurred.

Image stream 2172 is also input to a rate/resolution down-sampler 2162that simultaneously reduces the frame rate and resolution of imagestream 2172 to produce a live feed 2170. Still image feed 2166,slow-motion feed 2168 and live feed 2170 may be combined into a singledigital feed 2164. Camera control device 2104 may selectively modifydown-sampling by down-samplers 2154, 2158 and 2162 and selectivelytransfer images and image sequences from circular buffers 2155, 2159 tobuffers 2156, 2160, respectively, based upon algorithms 426 and/or inputfrom one or more of production control devices 614, 1414 and 1714, toprovided desired still images and image streams.

Camera 2110 may also include functionality of camera 2010, FIG. 20,through control and size of circular buffers 2155 and 2159, to alsoallow capture of future events of interest.

In an embodiment, imager 2152 includes a time stamp unit 2182 that issynchronized with a global clock 2180. Global clock 2180 provides a timesignal (e.g., including a time of day) to each component of camera 2110and camera control device 2104 to synchronize time within these andother devices of an autonomous still and/or moving picture productionsystem (see, e.g., system 1700, FIG. 17). Time stamp unit 2182 mayattach a time stamp to each frame of image stream 2172, or may attach atime stamp to images of image stream 2172 at an appropriate periodicity.Thus, as image stream 2172 is processed within camera 2110 and otherdevices of the system, capture time of the images may be determined.

Each notification 2130 received by camera control device 2104 may alsobe given a time stamp that determines when the notification occurred.Thus, based upon notifications and their associated time stamps, cameracontrol device 2104 may determine a period for the action of interestassociated with the notification 2130. Camera control device 2104 maythus send this period to camera 2110 to request still images and/or aslow-motion image stream of the action of interest. Camera 2110 may thentransfer captured images from circular buffers 2155, 2159 to buffers2156, 2160, respectively, based upon the requested period. In view ofthe continual operation of camera 2110, requests for still images and/orslow-motion image streams are preferably processed within camera 2110 inchronological order of the requested period, oldest first, and notnecessarily the order of request arrival at camera 2110, to avoidoverrun of circular buffers 2155 and 2159.

Although camera 2110 is shown with three exemplary image data paths(‘still image’, ‘slow-motion’, and ‘live’), camera 2110 may have more orfewer image data paths without departing from the scope hereof. In anembodiment, the number and type of image data paths (e.g., still image,slow-motion and live) operating within camera 2110 is configured bycamera control device 2104 and based upon the type of event beingcaptured. Rate down-sampler 2154, resolution down-sampler 2158 andrate/resolution down-sampler 2162 may be implemented as software moduleswithin one or more digital signal processors and thereby selected foroperation as required. Further, circular buffers 2155 and 2159, andoptionally buffers 2156 and 2160, may be sourced from a common memorydevice within camera 2110, and are thereby configurable in size andnumber by camera control device 2104. For example, camera control device2104 may configure camera 2110 with two slow-motion image data paths,each having a resolution down sampler, a rate down sampler and acircular buffer, for capturing slow-motion image streams at differentframe rates and/or different resolutions. That is, rate, resolution, andrate/resolution down samplers may be selectively combined to provide adesired image data path within camera 2110, and one or more such imagedata paths may be used simultaneously.

In one embodiment, camera 2110 has a limited bandwidth and is notcapable of running at its maximum resolution and maximum frame ratesimultaneously. Camera control device 2104 thus controls imager 2152 tooperate within constraints imposed by camera 2110.

FIG. 22 shows a schematic diagram 2200 of a production control device2214, a camera control device 2204, and two cameras 2210(1) and 2210(2).Camera control device 2104 may represent camera control devices 104,204, 304, 404, 504, 604, 1404, 1504 or 1704 of FIGS. 1, 2, 3, 4, 5, 6,14, 15 and 17, respectively. Cameras 2110 may each represent one or moreof cameras 110, 210, 210, 410, 510, 610, 1410, 1510 and 1710 of FIGS. 1,2, 3, 4, 5, 6, 14, 15 and 17, respectively. Production control 2214 mayrepresent one or more of production control devices 614, 1414 and 1714of FIGS. 6, 14 and 17, respectively.

Each camera 2210 sends a high resolution (e.g., 2048×1538 pixels) highframe rate (e.g., 120 frames per second) image stream 2272 to productioncontrol device 2214 where each image stream 2272 is processed by asignal processing channel 2202. Although FIG. 22 shows two signalprocessing channels 2202, production control device 2214 may includefewer or more signal processing channels 2202 without departing from thescope hereof.

High resolution high frame rate image stream 2272(1) connects to a ratedown-sampler 2254(1), a resolution down-sampler 2258(1) and arate/resolution down-sampler 2262(1). High resolution high frame rateimage stream 2272(2) connects to a rate down-sampler 2254(2), aresolution down-sampler 2258(2) and a rate/resolution down-sampler2262(2). Rate down-samplers 2254 reduce the frame rate of image streams2272. In one example of operation, rate down-sampler 2254(1) reduces theframe rate of image stream 2272(1) to five frames per second, and storesthese high-resolution still images within a still image buffer 2256(1).Resolution down sampler 2258(1) maintains the frame rate of image stream2272(1), but reduces the resolution of each frame and stores the reducedresolution high frame rate image stream in a slow-motion buffer 2260(1).Thus, an image stream 2268(1), when output from slow-motion buffer2260(1) at a frame rate lower than the high frame rate of image stream2272(1), appears to be slow motion while maintaining smooth imagesequencing. Rate/resolution down-sampler 2262(1) reduces the resolutionof each frame and reduces the frame rate of image stream 2272(1) toproduce image stream 2270(1). Thus, image stream 2270(1) may have astandard frame rate and standard resolution of a television feed.

Signal processing channel 2202(2) operates similarly to signalprocessing channel 2202(1) to process high resolution high frame rateimage stream 2272(2) from camera 2210(2), and will therefore not bedescribed further.

In particular, production control device 2214 controls each ofconverters 2254, 2258 and 2262 to produce desired outputs 2266, 2268 and2270, respectively. Production control device 2214 may then operate toselect between image streams 2266, 2268 and 2270 to provide one or moreoutput feeds 2205.

In one embodiment, each signal processing channel 2202 includes a buffer2003 that delays each image stream 2272 for a certain period (e.g.,thirty seconds), thereby allowing production control device 2214 toprocess and capture an event of interest identified by notifications2230 and occurring within that period. For example, where buffers 2003store thirty seconds of high resolution high speed image stream 2272 andproduction control device 2214 receives notifications 2230 indicatingthat an event of interest occurred twenty seconds ago, productioncontrol device 2214 determines that image stream relating to this eventwill be processed in ten seconds by each signal processing channel 2202and may thus store and utilize relevant still images and slow-motionimage streams for that event.

FIG. 23 is a schematic diagram illustrating one exemplary stadium 2301hosting a sporting event on field 2308 for which video and still imageproduction is provided by a system 2300 that includes four cameras2310(1), 2310(2), 2310(3) and 2310(4), an object tracking device 2302, acamera control device 2304, a production control device 2314, a databasecontrol device 2315 and a database 2318. Object tracking device 2302,camera control device 2304, production control device 2314, databasecontrol device 2315 and database 2318 may represent object trackingdevice 1702, camera control device 1704, production control device 1714,database control device 1715 and database 1718 of system 1700, FIG. 17.Production control device 2314 may include functionality of productioncontrol device 2214, FIG. 22. Cameras 2310 may represent cameras 110,210, 210, 410, 510, 610, 1410, 1510, 1710, 2010 and 2110 of FIGS. 1, 2,3, 4, 5, 6, 14, 15, 17, 20 and 21 respectively.

Within system 2300, camera control device 2304 and production controldevice 2314 cooperate to control cameras 2310 and generate feed 2305based upon tracking information received from object tracking device2302. In the example of FIG. 23, a player 2306 on fields 2308 has alocation device 2312 that provides location information to objecttracking device 2302. Camera control device 2304 and production controldevice 2314 determine that cameras 2310(1) and 2310(3) are mostappropriate to capture imagery of player 2306. Other players and objects(e.g., a ball) may be similarly tracked by system 2300, but are notshown for clarity of illustration.

System 2300 may also capture images within stadium 2301 at the requestof spectators or other interested parties. In one example, an externalinteraction device 2330 receives requests to image a location withinstadium 2301 from one of a plurality of request kiosks 2352 locatedthroughout stadium 2301. Kiosks 2352 communicate with externalinteraction device 2330 to request one or more still images and/ormoving images to be taken of a particular location. For example, aspectator 2307 utilizes one of kiosks 2352 to request a still image betaken of the spectator's seat location. The spectator may proffer aticket stub for kiosk 2352 to scan, for example, to input the seatlocation. External interaction device 2330 is programmed with locationsfor each seat within stadium 2301, and may therefore translate the inputseat number to a location relative to cameras 2310. Kiosk 2352 may allowthe spectator to input a time window during which the image is to betaken, thereby allowing the spectator time to return to the seat.External interaction device 2330 then interacts with camera controldevice 2304 and/or production control device 2314 to schedule anappropriate camera 2310 for imaging the requested camera within thescheduled time window. Where system 2300 utilizes cameras 2310 to imagesports activity and spectators, camera priority is preferably given tothe sports activity; cameras 2310 are used to image spectator locationswhen not required for imaging the sporting event. In the example of FIG.23, camera 2310(2) captures the image of seat location 2307. In anotherembodiment, system 2300 includes dedicated cameras for imagingspectators and may therefore schedule image capture times moreaccurately.

In another example, a spectator 2309 uses a cell phone 2311 withlocation capability (e.g., GPS) to request an image be taken of thecurrent location of spectator 2309. FIG. 23 shows camera 2310(4) imagingthe location of spectator 2309. External interaction device 2330 mayutilize a mapped topography of the stadium to determine a locationrelative to cameras 2310 based upon a received two-dimensionalcoordinate. Once the image it taken, a low-resolution image may be sentto the spectator's cell phone 2311, for example, together with a web URLand identification number that allows spectator 2309 to purchase theimage at a later time.

In another example, a spectator interacts with a web site to requestimages to be taken of a particular seat within the stadium. System 2300then takes one or more images of the specified seat location during theevent and makes the images available online. The spectator may then viewand purchase the images after returning from the stadium.

In another example, stadium 2301 advertises a number to which acell-phone may be used to text a seat number (e.g., a seat number andsection location) to request an image to be taken of that location. Whena spectator texts a seat number to the provided number, externalinteraction device 2330 determines when one of cameras 2310 suitable fortaking the image is available, and may notify the spectator of timewindow when the image will be taken. Alternatively, if system 2300 isbusy and no cameras 2310 are available to take the image, externalinteraction device 2330 may inform the spectator, by sending a textmessage to the spectator's phone, that the system is busy and that totry again later, or that they will be notified when the system is readyto take the image. The cost of sending the text message may include aninitial cost for taking the image. The captured image(s) may then beviewed online and purchased by the spectator.

System 2300 may record several still images to image database 2318 foreach requested location, thereby providing a choice to the requestor.

Kiosk 2352 may operate to allow the spectator to view a series ofcaptured imaged upon request, and thereby allow selection of one or moreimages for purchase. For example, purchased images may be emailed to aprovided email address. Optionally, kiosk 2352 may include aphotographic printer to print purchased images.

In one embodiment, system 2300 utilizes cameras 2310 to capture imagesof each seat within the stadium over the period of the sporting event.For example, system 2300 may capture images of seats within stadium 2301using cameras 2310 when they are not required for imaging the sportingevent itself. Each image being stored within image database 2318 with aseat number reference (or seat number range reference) to allowspectators to view and purchase the images online.

Before discussing FIGS. 24 and 25, FIG. 26 is a flowchart illustratingone exemplary method for capturing images of a location upon request.Method 2600 is for example implemented within one or more of cameracontrol devices 104, 204, 304, 404, 504, 604, 1404, 1504, 1704, 2004,2104, 2204 and 2304 of FIGS. 1, 2, 3, 4, 5, 6, 14, 15, 17, 20, 21, 22and 23, respectively, and production control device 614, 1414, 1714,2214 and 2314 of FIGS. 6, 14, 17, 22 and 23, respectively, and may beconsidered part of central control 2414, FIG. 24, described in detailbelow.

In step 2602, method 2600 receives an image capture request for alocation. In one example of step 2602, a spectator at a baseball gamesends a text message containing a seat and stand number to a designatedphone number monitored by external interaction device 2330 of FIG. 23.In another example of step 2602, a person accesses a web site thatinterfaces with external interaction device 2330 to request an image betaken of a certain seat location. In another example of step 2602, aspectator utilizes a cell phone to send latitude and longitudecoordinates, derived from a GPS unit within the cell phone, to thetelephone number to request an image of that location. In step 2604,method 2600 determines an optimal camera for capturing images of thelocation received in step 2602. In one example of step 2604, externalinteraction device 2330 converts the defined location received in step2602 into a location relative to cameras 2310 and field 2308 and sendsthe converted location to camera control device 2304 which determines atleast one of cameras 2310 that is optimal for capturing an image of theconverted location. In another example of step 2604, camera controldevice 2304 determines camera 2310(2) as the optimal camera forcapturing images of seat location 2307.

In step 2606, method 2600 determines availability of the optimal camerafor capturing the image. In one example of step 2606, camera controldevice determines that play on field 2308 is suspended by an injury andthat camera 2310(2) is available to capture images of seat location2307. In step 2608, method 2600 schedules the determined optimal cameraof step 2604 for capturing the image of the location of step 2602. Inone example of step 2608, production control device 2314 schedulescamera 2310(2) to capture images of seat location 2307 five minutesafter the request for image capture was received in step 2602. In oneembodiment, where the request for image capture of step 2602 wasreceived as a text message, production control 2314 may instructexternal interaction device 2330 to send a text message back to therequesting cell phone with the schedules time for the image to be taken.In step 2610, method 2600 waits for the scheduled capture time of step2608. In one example of step 2610, production control device 2314includes a scheduling table of events to process as certain times, whichis monitored while production control device 2314 continues normaloperation. In step 2612, method 2600 controls the determined optimalcamera of step 2604 to include the location of step 2602 within itsfield of view. In one example of step 2612, production control device2314 utilizes camera control device 2304 to control camera 2310(2) toposition seat location 2307 within its field of view. In step 2614,method 2600 captures at least one image of the location using thedetermined optimal camera. In one example of step 2614, productioncontrol device 2314 sends the at least one captured image to databasecontrol device 2315 which stores the at least one captured image withinimage database 2318. In step 2616, method 2600 delivers the image to thecapture requestor. In one example of step 2616, where the image capturerequest of step 2602 is received from a cell phone, external interactiondevice 2330 sends at lease a low-resolution image of the captured imageof step 2614 to the requesting cell phone. In another example of step2616, the captured image is made available on a web site accessed by theinternet 2340, for example referenced by seat number, thereby allowingthe requestor to view and purchase the image during or after thesporting event. In one example, the stored images are graphicallyannotated with information relative to the sporting event, such as thevenue, playing teams and event name. In another example of step 2616,kiosk 2352 (FIG. 23) is used to deliver one or more images to acustomer.

FIG. 24 shows a system 2400 with an exemplary central control unit 2414that represents intelligence of camera control devices 104, 204, 304,404, 504, 604, 1404, 1504, 1704, 2004, 2104, 2204 or 2304 of FIGS. 1, 2,3, 4, 5, 6, 14, 15, 17, 20, 21, 22 and 23, respectively and productioncontrol device 614, 1414, 1714, 2214 or 2314 of FIGS. 6, 14, 17, 22 and23, respectively. For example, central control unit 2414 representsintelligence of camera control device 2304 and production control device2314 of FIG. 23, where intelligence is divided between these devices.That is, functionality of central control unit 2414 may be implementedwithin camera control units and/or production control units withoutdeparting from the scope hereof. Central control unit 2414 providescontrol of one or more cameras (e.g., cameras 110, 210, 310, 410, 510,610, 1410, 1510, 1710, 2010, 2110, 2210 and 2310) to capture imagestreams that may include high-resolution still images and slow-motionimages and selects one or more of these images streams for producing animage stream feed (e.g., feed 105, 205, 405, 505, 605, 1405, 1705 and2205) and storing images within a database (e.g., image database 1518,1718, 1818, 2318).

Central control 2414 receives object location information 2416 thatincludes coordinate data (e.g., coordinate data 116) from objecttracking devices (e.g., object tracking devices 102, 202, 302, 402, 502,602, 1402, 1502, 1702, 2002, 2102, 2202 or 2302) and from other dynamicobjects that affect camera field of view selection, such as the positionof the sun, thereby allowing central control unit 2414 to have dynamicobject awareness 2452. Central control unit 2414 also receivescoordinate data for static objects that may fall into a field of view ofone or more cameras, such as columns, pillars, the sporting event field,goal posts, and other static objects. For example, central control unit2414 may utilize a topographical map of a stadium. In another example,central control unit 2414 utilizes a map of a track and field event,thereby forming object awareness of individual activities, such as highjump, pole vault, running track, etc. Central control unit 2414 therebyhas static object awareness 2454. Central control unit 2414 may thusdetermine an optimal field of view from each camera based upon dynamicand static object location relative to the camera.

Central control unit 2414 may receive other information relating to theproduction of a video feed for the event being captured. Central controlunit 2414 receives notifications 2430 (e.g., notifications fromnotification device 1730, FIG. 17) that may be significant to capture ofinteresting image streams and production of an image feed (e.g., feed105, 205, 405, 505, 605, 1405, 1705 or 2205). As noted above,notifications 2430 may automatically result from score changes on ascoreboard, may be generated by user input and other means. Centralcontrol unit 2414 includes a notification reaction module 2456 thatutilizes notifications 2430 to identify events of interest and assignone or more cameras to those events and include image streams thatcapture the event within the output feed. In one example, notifications2430 may include status of a game clock, such that central control unit2414 may determine if play is in progress, and where play is not inprogress, central control unit 2414 may utilize other notifications andobject location information to determine areas of interest for captureusing one or more cameras. For example, if the game clock has stoppedand a coach is near a game official, central control unit 2414 mayassign one or more cameras to capture an image stream of the coach andofficial.

Central control unit 2414 may also receive statistical data 2458 thatincludes statistics on one or more players or teams in a sporting event.Statistical data 2458 may be used by a central intelligence 2460 ofcentral control unit 2414 to identify one or more players of interestsuch image feeds of these players are captured more frequently. In oneexample, statistical data 2458 includes statistics for an athlete thatis close to breaking a record and therefore central intelligence 2460decides to give higher priority to assigning a camera to that athlete,thereby capturing an image stream (and/or still images) of events thatmay result in the record being broken.

Central control unit 2414 may also receive an event schedule 2462 thatprovides details and times of certain events of interest within asporting event or production. For example, event schedule 2462 mayprovide race times for a track and field event, thereby allowing centralintelligence 2460 to determine appropriate times when each part of thefield will be used, particularly since there may be many trackedathletes within the event field at any one time. Central control unit2414 utilizes event schedule 2462 to determine when each event (e.g., atrack race) finishes and when a next event is due to start, therebyallowing central control unit 2414 to coordinate coverage of each aspectof the event or production. Where there is a period of inactivity, forexample between scheduled races of a track and field event, centralcontrol unit 2414 may elect to play back image streams of previouslyrecorded events, for example showing final stages of a race in slowmotion.

Central control unit 2414 may also receive sport specific information2464 that allows an event awareness module 2466 to determine when eventsof interest may occur for a particular sporting event. For example,sport specific information 2462 may define events of interest for abaseball game, such as when a runner of the team at bat moves away fromfirst base (e.g., when trying to steal second base), central controlunit 2414 may assign a camera to the runner to ensure no event ofinterest is missed. Further, the sport specific information 2464 mayspecify that a pitcher standing on the mound of a baseball game is ofinterest, as is the player at bat when within a certain distance of homeplate. In this example, central control unit 2414 may operate to displaythe image stream of the runner on first base within a picture-in-pictureof the main feed. In another example, where central control unit 2414 ispart of a system imaging an American football game, there may be severalfootballs equipped with a location tag. However, by including a rulewith sport specific information 2464 that specifies that during play(i.e., when the game clock is running) only one football is on the fieldof play, central control unit 2414 may determine which of many footballsis of interest. Central intelligence 2460 may utilize event awarenessmodule 2466, sport specific information 2464 and dynamic objectawareness 2452 to determine a position for an imaged target within acamera's field of view. For example, when tracking and capturing animage stream of a football player running with the ball, the position ofthe player within the camera field of view may be selected such that thecamera ‘leads’ or anticipates the player such that a viewer may seeopposition players who may tackle the player. In a horse race, on theother hand, it may be desirable to position the lead horse by making thecamera ‘lag’ such that a view may see other horses that may catch thelead horse.

Central intelligence 2460 utilizes information processed by modules2452, 2454, 2456 and 2466 and applies this combined intelligence toproduce a live feed (e.g., feed 105, 205, 405, 505, 605, 1405, 1705 and2205) of an event or production.

Central control 2414 may also include a graphic generator 2468 thatproduces generated graphics 2470 containing statistical data for playersand teams, based upon statistical data 2458 for example, and may includeresult information where this information is received by central control2414. Central control 2414 may utilize sport specific information 2464to collect statistical information for a sporting event. For example,sport specific information 2464 may specify that a graphic of teampossession time may be generated for an American football game basedupon statistics accumulated by central control 2414; the graphic maythen be selectively included within an image stream feed by centralcontrol 2414 at appropriate times between events of interest and gameplay. A similar metric may be applied to individuals in a soccer game.In another example, graphic generator 2468 generates a position table ofparticipants in a race in real-time. Such automatically generated tablesmay also be fed to and used with other systems, such as standard TVproduction. without departing from the scope hereof.

Decisions made by central control 2414 are based upon a number ofavailable cameras that are controlled by central control 2414 and thenumber of image stream feeds generated by central control 2414. Forexample, where central control 2414 controls a system with more cameras,central control 2414 may operate to use cameras to follow individualplayers/athletes; where central control 2414 controls a system withfewer cameras, central control 2414 may operate to utilize these fewercameras more selectively to generate an interesting image stream feed.

Each camera control device 104, 204, 304, 404, 504, 604, 1404, 1504,1704, 2004, 2104, 2204 and 2304 of FIGS. 1, 2, 3, 4, 5, 6, 14, 15, 17,20, 21, 22 and 23, respectively, and production control device 614,1414, 1714, 2214 and 2314 of FIGS. 6, 14, 17, 22 and 23, respectively,may also include an audio feed that includes audio of the captured imagestream. For example, one or more cameras 110, 210, 310, 410, 510, 610,1410, 1510, 1710, 2010, 2110, 2210 and 2310 may include microphones suchthat a captured audio signal is sent from the camera to the cameracontrol device and/or production control device. In another example, oneor more microphones are located within a stadium to capture audio feeds.These audio feeds are received by the camera control devices and/orproduction control devices and may be recorded (e.g., by recordingdevices 220, 320, 1418 and/or database control devices 1515, 1715 and2315) in association with recorded image streams. In one example,central control 2414 utilizes notifications 2430 and sport specificinformation 2464 to capture an audio stream of official announcementsduring a sporting event.

Although not shown accompanying image streams in FIGS. 1, 2, 3, 4, 5, 6,14, 15, 17, 19, 23 and 25, audio may accompany stored image streams andfeeds without departing from the scope hereof. For example, audiocaptured by one or more microphones located around a stadium may bemixed to provide an audio feed to accompany image feeds.

FIG. 25 shows once exemplary system 2500 for including commentary with avideo feed 2562. A production control device 2514 produces an imagestream 2505, from one or more image feeds 2519, for display on a displaydevice 2520. Production control device 2514 may also send annotationdata 2527 and statistical data 2558 to display device 2520 for displayin association with image stream 2505. System 2500 includes a microphone2552 for capturing commentary 2560 relating to image stream 2505 by acommentator 2530. Annotation data 2527 may list players shown withinimage stream 2505. Statistical data 2558 may include statistics for eachof these players and their teams' performance figures. In particular,since production control device 2514 is aware of content of image stream2552, annotation data 2527 and statistical data 2558 may be selectivelydisplayed with relevance to image stream 2505, thereby providingcommentator 2530 with relevant information for use within commentary2560.

Image stream 2505, commentary 2560 and annotation data 2527 may bestored by a database control device 2515 within an image database 2518and commentary 2560 may be stored within an audio database 2519. In oneembodiment, image database 2518 and audio database 2519 are part of thesame database. Image stream 2505 and commentary 2560 may be combined andoutput as a live feed 2562 (e.g., a TV feed). As appreciated, commentary2560 may also be mixed with other audio received by production controldevice 2514 without departing from the scope hereof.

FIG. 27 shows one exemplary commentary device 2750 for automaticallyadding commentary 2760 to an automatically produced video feed 2762.Commentary device 2750 includes a commentary generator 2752 and a voicesynthesizer 2754. Commentary generator 2752 processes annotation data2727, statistic information 2758, and operational data 2764 to generateappropriate commentary 2760 for the generated live feed 2762.Operational data 2764 may indicate one or more of selected image streamfor output as live feed 2762, events of interest, predicted image streamselection, and other information relevant to production of live feed2762. Production control device 2714 may include buffers to delay imagestreams 2719 (see for example FIG. 22 and associated description)thereby allowing commentary 2760 to be generated for events 2730 thatindicate already occurred events of interest.

In one example of operation, each player in an American football gamewears at least one location device, and each football used during thegame includes a location device. Production control device 2714 receivesannotation data 2727 and is thereby able to determine which player iswithin each image stream, based upon field of view information for eachimage stream 2719, which image stream includes the football and whichplayer has control of the ball (e.g., by proximity and motion of boththe player and the football). Thus, using sport specific information(e.g., sport specific information 2464, FIG. 24) commentary generator2752 may provide interesting and accurate commentary of a sportingevent. Continuing with this example, as a quarterback throws thefootball, central control 2414 may determine trajectories of thefootball and players to predict the outcome to change cadence and pitchof the synthesized voice of voice synthesizer 2754.

In another example, where production control device 2714 utilizesgraphics, appropriate commentary 2760 may be generated by feeding thestatistical and/or graphical information to commentary generator 2752.

FIG. 29 is a high-level block diagram 2900 illustrating exemplaryhardware of an object tracking device 2902, a camera control device2904, a camera 2910, a production control device 1914 and a databasecontrol device 2915. Object tracking device 2902, camera control device2904, camera 2910, production control device 2914 and database controldevice 2915 may represent hardware of object tracking devices, cameracontrol devices, cameras, production control devices and databasecontrol devices of FIGS. 1, 2, 3, 4, 5, 6, 14, 15, 17, 20, 21, 22 and23.

Object tracking device 2902 is shown with a processor 2952, a memory2954 and an interface 2955. In an embodiment, interface 2955 is awireless interface for communicating with and/or receiving data from oneor more location units (e.g., location units 112). Processor 2952 andmemory 2954 facilitate processing of received information and transferof this information to camera control device 2904.

Camera control device 2904 is shown with a processor 2956 and a memory2958 that facilitate implementation of algorithms 426 (FIG. 4) and/orfunctionality of at least part of central control unit 2414, FIG. 24.

Camera 2910 is shown with an imager 2968, a processor 2970 and a memory2972 that facilitate implementation of functionality of cameras 2010 and2110 (FIGS. 20 and 21, respectively). Camera 2910 may include additionalprocessors, such as digital signal processors and memory withoutdeparting from the scope hereof.

Production control device 2914 is shown with a processor 2960 and amemory 2962 that facilitate implementation of at least part of centralcontrol unit 2414.

Database control device 2915 is shown with a processor 2964 and a memory2966 that facilitate management of one or more databases and/or otherstorage devices.

FIG. 30 is a flowchart illustrating one exemplary method 3000 forselectively capturing a standard feed, still images and a slow-motionfeed within camera 2010 of FIG. 20.

Method 3000 may be implemented within a processor (e.g., processor 2970,FIG. 29) of camera 2010. FIGS. 20, 30, 31, 32, 33 and 34 are best viewedtogether with the following description.

Step 3002 is an initialization step. In step 3002, method 3000 sets thecamera imager (e.g., imager 2052) to capture images at a standardresolution (e.g., 640×480 pixels) and at a standard frame rate (e.g.,thirty frames per second), and then sets the rate/resolutiondown-sampler (e.g., rate/resolution down-sampler 2062) to notdown-sample. That is, since imager 2052 is capturing images at theresolution and frame rate of live feed 2070, no down sampling isrequired. In step 3004, method 3000 receives a capture command. In oneexample of step 3004, camera 2010 receives a command to capture stillimages. Step 2006 is a decision based upon the command action receivedin step 3004. If the received capture command of step 3004 commandsslow-motion on, method 3000 continues with step 3008; if the receivedcapture command of step 3004 commands slow-motion off, method 3000continues with step 3010; if the received capture command of step 3004commands still picture on, method 3000 continues with step 3012; and ifthe received capture command of step 3004 commands still picture off,method 3000 continues with step 3014.

In step 3008, method 3000 calls a sub-method 3100, shown in FIG. 31.Upon return from sub-method 3100, method 3000 continues with step 3004.In step 3010, method 3000 calls a sub-method 3200, shown in FIG. 32.Upon return from sub-method 3200, method 3000 continues with step 3004.In step 3012, method 3000 calls a sub-method 3300, shown in FIG. 33.Upon return from sub-method 3300, method 3000 continues with step 3004.In step 3014, method 3000 calls a sub-method 3400, shown in FIG. 34.Upon return from sub-method 3400, method 3000 continues with step 3004.

In step 3102, sub-method 3100 sets the imager to capture at a fast framerate. In one example of step 3102, processor 2970 sets imager 2052 tocapture images at a frame rate of one-hundred and twenty frames persecond. Step 3104 is a decision. If, in step 3104 sub-method 3100determines that the camera is also operating in still image mode,sub-method 3100 continues with step 3106; otherwise sub-method 3100continues with step 3112.

In step 3106, sub-method 3100 sets the rate down-sampler to reduce theframe rate from the fast frame rate. In one example of step 3106,processor 2970 sets rate down-sampler 2054 to reduce the frame rate ofimage stream 2070 from one-hundred and twenty frames per second to fiveframes per second. In step 3108, sub-method 3100 sets the resolutiondown sampler to reduce image resolution from the high resolution usedfor still images to the standard resolution of the slow-motion imagestream. In one example of step 3108, processor 2970 sets resolution downsampler 2058 to reduce the resolution of each frame of image stream 2072to a standard resolution from the high resolution (e.g., 2048×1536pixels) used to produce the still images. In step 3110, sub-method 3100sets the rate/resolution down-sampler to reduce the frame rate from thefast frame rate set in step 3102 to a standard frame rate and to reducethe high resolution to a standard resolution. In one example of step3110, processor 2970 sets rate/resolution down-sampler 2062 to convertsa captured image stream 2072 resolution of 2048×1536 pixels to astandard resolution of 640×480 pixels and to converts a captured fastframe rate of one-hundred and twenty frames per second to a standardframe rate of thirty frames per second. Sub-method 3100 then continueswith step 3116.

In step 3112, sub-method 3100 sets resolution down sampler to not reducethe standard resolution of the captured image stream. In one example ofstep 3112, processor 2970 sets resolution down-sampler 2058 to notreduce resolution of each frame of image stream 2072. In step 3114,sub-method 3100 sets the rate/resolution down-sampler to reduce theframe rate from the fast frame rate set in step 3102. In one example ofstep 3114, processor 2970 sets rate/resolution down-sampler 2062 toreduce the frame rate of image stream 2072 from the fast frame rate ofone-hundred and twenty frames per second to a standard frame rate ofthirty frames per second.

In step 3116, sub-method 3100 transfers the slow-motion image stream tothe slow-motion image stream buffer. In one example of step 3116,processor 2970 transfers slow-motion image stream from resolutiondown-sampler 2058 to slow-motion buffer 2060, from where it is output asslow-motion image stream 2068. Sub-method 3100 then returns to step 3004of method 3000.

In step 3202, sub-method 3200 sets the imager to capture images at astandard frame rate. In one example of step 3202, processor 2970 setsimager 2052 to capture image stream 2072 at thirty frames per second.Step 3204 is a decision. If, in step 3204, sub-method 3200 determinesthat the camera is also operating to capture still images, sub-method3200 continues with step 3206; otherwise sub-method 3200 continues withstep 3210.

In step 3206, sub-method 3200 sets rate down-sampler to reduce the framerate from the standard frame rate set in step 3202. In one example ofstep 3206, processor 2970 sets rate down sampler 2054 to reduce theframe rate of image stream 2072 from thirty frames per second to fiveframes per second. In step 3208, sub-method 3200 sets therate/resolution down-sampler to not reduce the frame rate and to reducethe resolution from high resolution used by still picture capture to astandard resolution. In one example of step 3208, processor 2970 setsrate/resolution down-sampler 2062 to not reduce the frame rate of imagesteam 2072 and to reduce the resolution of each frame of image stream2072 to the standard resolution of 640×480 pixels. Sub-method 3200continues with step 3212.

In step 3210, sub-method 3200 sets the rate/resolution down-sampler tonot reduce the frame rate and to not reduce resolution. In one exampleof step 3210, processor 2970 turns rate/resolution down-sampler 2062 offsuch that image stream 2072 passes through to become image feed 2070without change.

In step 3212, sub-method 3200 sets the resolution down-sampler off. Inone example of step 3212, processor 2970 sets resolution down sampler2058 off as no slow-motion image stream 2068 is required. In step 3214,sub-method 3200 stops transfer of slow-motion image stream fromresolution down-sampler 2058 to slow-motion buffer 2060. Sub-method 3200then returns to step 3004 of method 3000.

In step 3302, sub-method 3300 sets the imager to capture at a highresolution. In one example of step 3302, processor 2970 sets imager 2052to capture at 2048×1536 pixels using the previously set frame rate. Step3304 is a decision. If, in step 3304, sub-method 3300 determines thatthe camera is also operating to capture a slow-motion image stream,sub-method 3300 continues with step 3306; otherwise sub-method 3300continues with step 3312. In step 3306, sub-method 3300 sets theresolution down-sampler to reduce the resolution of each frame from thehigh-resolution set in step 3302 to a standard resolution of theslow-motion image stream. In one example of step 3306, processor 2970sets resolution down-sampler 2058 to reduce the resolution of each frameof image stream 2072 from 2048×1536 pixels to a standard resolution of640×480 pixels. In step 3308, sub-method 3300 sets the rate down-samplerto reduce the frame rate of the captured image stream from the fastframe rate of the slow-motion image stream to the frame rate of thestill picture image stream. In one example of step 3308, processor 2970sets rate down-sampler 2054 to reduce the frame rate of image stream2072 from one-hundred and twenty frames per second to 5 frames persecond. In step 3310, sub-method 3300 sets the rate/resolutiondown-sampler to reduce the frame rate from the fast frame rate to astandard frame rate and to reduce the resolution from thehigh-resolution set in step 3302 to a standard resolution. In oneexample of step 3310, processor 2970 sets rate/resolution down-sampler2062 to reduce the frame rate of image stream 2072 from one-hundred andtwenty frames per second to thirty frames per second and to reduce theresolution of each remaining frame of image steam 2072 from 2048×1536pixels to 640×480 pixels. Sub-method 3300 continues with step 3314.

In step 3312, sub-method 3300 sets the rate/resolution down-sampler toreduce the resolution from the high-resolution set in step 3302. In oneexample of step 3312, processor 2970 sets rate/resolution down-sampler2062 to reduce the resolution of each frame of image stream 2072 from2048×1536 pixels to 640×480 pixels while leaving the frame rateunchanged.

In step 3314, sub-method 3300 transfers still pictures to the stillpicture buffer. In one example of step 3314, processor 2970 transfersstill images from rate down-sampler 2054 to still image buffer 2056.Sub-method 3300 then returns to step 3004 of method 3000.

In step 3402, sub-method 3400 sets the imager to capture images at astandard resolution. In one example of step 3402, processor 2970 setsimager 2052 to capture image stream 2072 at a standard resolution of640×480 pixels. Step 3404 is a decision. If, in step 3404, sub-method3400 determines that the camera is also operating to capture aslow-motion image stream, sub-method 3400 continues with step 3406;otherwise sub-method 3400 continues with step 3410.

In step 3406, sub-method 3400 sets resolution down-sampler to not reducethe resolution from the standard resolution set in step 3402. In oneexample of step 3406, processor 2970 sets resolution down sampler 2058to not reduce the resolution of image stream 2072. In step 3408,sub-method 3400 sets the rate/resolution down-sampler to reduce theframe rate from the fast frame rate used for the slow-motion imagestream to a standard frame rate and to not reduce the resolution. In oneexample of step 3408, processor 2970 sets rate/resolution down-sampler2062 to reduce the frame rate of image steam 2072 from one-hundred andtwenty frames per second to a standard frame rate of thirty frames persecond and to not reduce the resolution of each frame of image stream2072. Sub-method 3400 continues with step 3412.

In step 3410, sub-method 3400 sets the rate/resolution down-sampler tonot reduce the frame rate and to not reduce resolution. In one exampleof step 3410, processor 2970 sets rate/resolution down-sampler 2062 offsuch that image stream 2072 passes through to become image feed 2070without change.

In step 3412, sub-method 3400 turns the rate down-sampler off. In oneexample of step 3412, processor 2970 sets rate down sampler 2054 off asno still picture stream 2066 is required. In step 3414, sub-method 3400stops transfer of still images from rate down-sampler 2054 to stillimage buffer 2056. Sub-method 3400 then returns to step 3004 of method3000.

As appreciated, frame rates and resolutions shown in the above examplesmay vary without departing from the scope hereof.

FIG. 35 is a plan view 3500 of an operational field 3508 (e.g., a soccerfield) with four fixed cameras 3510(1), 3510(2), 3510(3) and 3510(4)positioned at corners of operational field 3508 and each having a fixedfield of view 3520 to capture images of activities within operationalfield 3508. For clarity of illustration, only field of view 3520 ofcamera 3510(3) is shown in FIG. 35. As appreciated, fewer or morecameras 3510 may be used without departing from the scope hereof. Fourtracked objects of interest 3506(1), 3506(2), 3506(3), 3506(4) and 3507are shown within operational field 3508 and captured by cameras 3510.Objects 3506 may represent soccer players and object 3507 may representa soccer ball. Cameras 3510 are high resolution (e.g., 10,000×7,500pixels) cameras that may be used with, or in place of, cameras 110, 210,210, 410, 510, 610, 1410, 1510, 1710, 2010 and 2110 of FIGS. 1, 2, 3, 4,5, 6, 14, 15, 17, 20 and 21, respectively, to generate image streamsand/or still images.

FIG. 36 shows one exemplary perspective view 3600 from camera 3510(3)and containing objects 3506 and 3507. Since object 3506(1) is closer tocamera 3510(3) than object 3506(4), object 3506(1) appears larger withinview 3600 than object 3506(4), when objects 3506(1) and 3506(4) aresimilar in size.

A camera control device (not shown) utilizes the location and fixedfield of view of each of cameras 3510 to determine one or more windows3652 within view 3600 (shown in dashed lines). Window 3652(1), shownenclosing object 3506(1), and window 3652(2), shown enclosing object3506(4), are of differing sizes and thus contain a different number ofpixels. Each window may be determined by the camera control device tocapture an image stream of a particular object (e.g., window 3652(1)captures object 3506(1)), based upon location information of each object3506. Windows 3652(1) and 3652(2) may be determined by the cameracontrol device in a manner similar to the determination of the field ofviews described above.

Where view 3600 represents an image with a resolution of 10,000×7,500pixels, window 3652(1) may have a resolution of 2800×2100 pixels andwindow 3652(2) may have a resolution of 640×480 pixels. The imageswithin each window may be resized to produce a consistent image streamfrom each window, particularly where multiple windows are used for onecamera. For example, where an output stream with a resolution of 640×480pixels is desired, each frame obtained from window 3652(1) may beconverted (e.g., down-sampled or up-sampled) to 640×480 pixels. Sinceeach window may change size, the ratio of this conversion is dynamic toprovide a constant output resolution.

In another example, where only one window is used for each camera, awindowing feature of an imager within the camera may be used to capturean image stream containing only the window contents. Thus, the windowimage need not be ‘cut’ from the larger view 3600.

Changes may be made in the above processes and systems without departingfrom the scope hereof. It should thus be noted that the matter containedin the above description or shown in the accompanying drawings should beinterpreted as illustrative and not in a limiting sense. For example,although fully automated production is shown in many of the aboveexamples, production control devices 614, 1414, 1714, 2214 and 2314 ofFIGS. 6, 14, 17, 22 and 23, respectively, may provide displays ofreceived image streams to allow a user to select the appropriate imagestream for output. The following claims are intended to cover allgeneric and specific features described herein, as well as allstatements of the scope of the present process and system, which, as amatter of language, might be said to fall there between.

What is claimed is:
 1. An image-stream windowing method, comprising: capturing, with a camera located at a fixed position and having a fixed field of view, a high-resolution image stream of an object that moves during said capturing, the high-resolution image stream comprising a sequence of high-resolution frames; determining, for each high-resolution frame of the sequence of high-resolution frames, a respective window, of a sequence of windows corresponding to the sequence of high-resolution frames, that encloses the object within said each high-resolution frame, said determining being based at least on the fixed position, the fixed field of view, and a position of the object; and generating a low-resolution image stream from the high-resolution image stream by cropping said each high-resolution frame with its respective window to create a respective one of a sequence of cropped frames, the low-resolution image stream comprising the sequence of low-resolution frames.
 2. The image-stream windowing method of claim 1, further comprising outputting the low-resolution image stream.
 3. The image-stream windowing method of claim 1, wherein a resolution of the respective window changes over the sequence of windows.
 4. The image-stream windowing method of claim 1, further comprising converting each cropped frame of the sequence of cropped frames into a respective one of a sequence of fixed-resolution frames such that all of the fixed-resolution frames have an identical resolution that is less than a resolution of the high-resolution image stream; wherein the low-resolution image stream comprises the sequence of fixed-resolution frames.
 5. The image-stream windowing method of claim 1, further comprising determining the position of the object during said capturing.
 6. The image-stream windowing method of claim 5, wherein said determining the position of the object includes determining coordinate data of the object using a wireless tracking tag affixed to the object.
 7. The image-stream windowing method of claim 6, wherein said determining the coordinate data of the object includes determining two spatial coordinates of the object.
 8. The image-stream windowing method of claim 6, wherein said determining the coordinate data of the object includes using multilateration of a radiofrequency signal transmitted by the wireless tracking tag.
 9. The image-stream windowing method of claim 1, wherein: said capturing includes capturing the high-resolution image stream of a first object and a second object that both move during said capturing; said determining includes: determining, for said each high-resolution frame, a respective first window, of a sequence of first windows corresponding to the sequence of high-resolution frames, that encloses the first object within said each high-resolution frame, said determining the respective first window being based at least on a first position of the first object; and determining, for said each high-resolution frame, a respective second window, of a sequence of second windows corresponding to the sequence of high-resolution frames, that encloses the second object within said each high-resolution frame, said determining the respective second window being based at least on a second position of the second object; and said generating includes: generating a first low-resolution image stream from the high-resolution stream by cropping said each high-resolution frame with its respective first window to create a respective one of a first sequence of cropped frames; and generating a second low-resolution image stream from the high-resolution stream by cropping said each high-resolution frame with its respective second window to create a respective one of a second sequence of cropped frames.
 10. The method of claim 9, further comprising outputting one or both of the first and second low-resolution image streams.
 11. An image-stream windowing system, comprising: a camera control device operable to: receive, from a camera located at a fixed position and having a fixed field of view, a high-resolution image stream of an object that moves while the camera captures the high-resolution image stream, the high-resolution image stream comprising a sequence of high-resolution frames, determine, for each high-resolution frame of the sequence of high-resolution frames, a respective window, of a sequence of windows corresponding to the sequence of high-resolution frames, that encloses the object within said each high-resolution frame, the camera control device being operable to determine the respective window based at least on the fixed position, the fixed field of view, and a position of the object, and generate a low-resolution image stream from the high-resolution image stream by cropping said each high-resolution frame with its respective window to create a respective one of a sequence of cropped frames, the low-resolution image stream comprising the sequence of low-resolution frames.
 12. The image-stream windowing system of claim 11, the camera control device being further operable to output the low-resolution image stream.
 13. The image-stream windowing system of claim 11, wherein a resolution of the respective window changes over the sequence of windows.
 14. The image-stream windowing system of claim 11, wherein: the camera control device is further operable to convert each cropped frame of the sequence of cropped frames into a respective one of a sequence of fixed-resolution frames such that all of the fixed-resolution frames have an identical resolution that is less than a resolution of the high-resolution image stream; and the low-resolution image stream comprises the sequence of fixed-resolution frames.
 15. The image-stream windowing system of claim 11, the camera control device being further operable to receive the position of the object from an object tracking system.
 16. The image-stream windowing system of claim 11, further comprising the object tracking system.
 17. The image-stream windowing system of claim 16, wherein: the object tracking system comprises a wireless tracking tag that is configured to be affixed to the object and operable to transmit a periodic radiofrequency signal; the object tracking system is operable to determine the position of the object by performing multilateration of the periodic radiofrequency signal.
 18. The image-stream windowing system of claim 11, the camera control device being further operable to: receive, from the camera, the high-resolution image stream of a first object and a second object that both move while the camera captures the high-resolution image stream, determine, based at least on a first position of the first object, a respective first window of a sequence of first windows corresponding to the sequence of high-resolution frames, the respective first window enclosing the first object within said each high-resolution frame, determine, based at least on a second position of the second object, a respective second window of a sequence of second windows corresponding to the sequence of high-resolution frames, the respective second window enclosing the second object within said each high-resolution frame, generate a first low-resolution image stream from the high-resolution stream by cropping said each high-resolution frame with its respective first window to create a respective one of a first sequence of cropped frames, and generate a second low-resolution image stream from the high-resolution stream by cropping said each high-resolution frame with its respective second window to create a respective one of a second sequence of cropped frames.
 19. The image-stream windowing system of claim 11, the camera control device being further operable to output one or both of the first and second low-resolution image streams.
 20. The image-stream windowing system of claim 11, further comprising the camera. 